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ORGANIZATION OF JOINT CHIEFS OF STAFF 
AND SYSTEMS ACQUISITION: 

WHAT NOW/WHAT NEXT 


Lieutenant Colonel Christopher A. Wain, USAF 



An informal photograph of the Joint Chief of Staff. Left to right: General Alfred M. Gray, USMC; Admiral Carlisle A. H. Trost, USN; General Larry D. Welch, 
USAF; General Carl Edward Vuono, USA; Admiral William J. Crowe, Jr., USN, chairman; and General Robert T. Hems, USAF. 


D efense acquisition process reform is a cottage in¬ 
dustry in Washington, D.C., and 1986 was a 
banner year. Two events occurred which have changed the 
manner in which the defense acquisition bureaucracy 
functions—passage of the Defense Reorganization Act of 
1986 and issuance of National Security Decision Directive 
(NSDD) 219. 

These two events profoundly affected the Office of the 
Secretary of Defense (OSD), the Services, and the Organiza¬ 
tion of the Joint Chiefs of Staff (OJCS). This article defines 
the role the OJCS now plays and can be expected to play 
in the acquisition process and summarizes the potential im¬ 
pact on the pi-ogram manager. 

The OJCS was reorganized in 1986, and so terms of 
reference should be redefined even for those who have 
served there. The OJCS consists of the Chairman, Joint 
Chiefs of Staff (CJCS), the Vice Chairman, Joint Chiefs of 
Staff (VCJCS), and their personal staffs; the Joint Chiefs of 
Staff (JCS); the Joint Staff, and the individual agencies and 
boards sporting thereto. The Joint Staff now supports the 
CJCS in his role as the independent military advisor to the 
Secretary of Defense (SECDEF), the National Security Coun¬ 
cil, and President. 


Vice Chairman, Joint Chiefs of Staff, is a new position 
created by the Reorganization Act with responsibilities de¬ 
fined by the CJCS and approved by the SECDEF. The 
VCJCS is the second-ranking military member of the Depart¬ 
ment of Defense and, by CJCS option, the focal point for 
OJCS involvement in systems acquisition. 

In general, the VCJCS is responsible for seeing that CJCS 
goals—improving the way joint system requirements are 
identified, justified, analyzed, and satisfied and improving 
the way joint programs are managed—are achieved. Dur¬ 
ing the long-term, the VCJCS will be the thread of continuity 
from CINC requirements to Service Program Objective 
Memorandums. 

These goals are not new. In 1984, the Defense Science 
Board identified the need for a senior group to look after 
the requirements for, and management of, joint programs. 
Shortly thereafter, the JCS chartered the Joint Requirements 
and Management Board (JRMB) which did yeoman service 
during its first two years of operation. The original JRMB 
consisted of the Vice Chiefs of the Army and Air Force, The 
Vice Chief of Naval Operations the Assistant Commandant 
of the Marine Corps, and the Director, Joint Staff. Chair¬ 
manship was rotated annually among Service members. 
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More than $4.5 billion in cost 
avoidance were documented as a result 
of JRMB efforts to identify oppor¬ 
tunities for jointness in systems 
acquisition. 

Pragmatically, JRMB efforts focused 
more on the reduction of duplication 
of effort in acquisition programs 
underway than on melding individual 
Service requirements into joint ones. 
Whatever its focus, the JRMB was a 
quiet, insistent, and effective force for 
giving the taxpayers more for their 
dollars; so effective, in fact, that the 
Packard Commission seized upon the 
JRMB name to replace the name (and 
to some extent, the function) of the 
Defense Systems Acquisition Review 
Council (DSARC). Accordingly, in 
June 1986, the original JRMB was 
renamed the Joint Requirements Over¬ 
sight Council (JROC); its charter was 
not changed at this time. 

Before 1984, the OJCS usually pro¬ 
vided a senior representative to 
DSARC deliberations. Participation in 
the acquisition decision process was 
largely ad hoc, however, because there 
was no bureaucratic infrastructure 
within the Joint Staff to support any 
greater level of involvement. Such in¬ 
volvement was not envisioned when 
OJCS was created and staffed. This has 
given rise to a definitely outdated and 
unnecessarily broad criticism within 
the acquisition bureaucracy that "the 
JCS never met a requirement it didn't 
like." 

This situation was in remedy during 
the same period as the early JROC ac¬ 
tivity. The Strategic Plans and 
Resource Analysis Agency (SPRAA), 
created at the behest of the Congress, 
gave the CJCS something of a program 
analysis and evaluation capability. The 
SPRAA was staffed with officers who 
understood both the acquisition pro¬ 
cess and the Programming, Planning, 
and Budgeting System (PPBS). The 
OJCS attendees at DSARC meetings 
began to arrive more fully prepared to 
deal with the issues, but a continuous 
thread to OJCS involvement in the 
process was absent. 


When the first VCJCS, General 
Robert T. Herres of the U. S. Air Force 
arrived, many changes were imminent. 
The NSDD 219, implementer for the 
Packard Commission recommenda¬ 
tions, had just been received for action 
and the Joint Staff had been restruc¬ 
tured in anticipation of the Chairman's 
changing role. Mr. Richard Godwin, 
had just assumed the newly created 
position of Under Secretary of Defense 
for Acquisition (USD(A)). 


JL he DODD 
5000.49 created the 
Defense Acquisition 
Board and ten com¬ 
mittees were created 
to replace the 126 
boards and councils 
that had carved out 
their own niches in 
the bureaucratic 
cliff. 
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Considerable activity ensued in the 
weeks and months following their ar¬ 
rivals. The DOD Directive 5000.1 and 
DODI 5000.2 were rewritten and 
staffed. The DODD 5000.49 created 
the Defense Acquisition Board (DAB), 
(a successor to the JRMB, aka 
DSARC), and ten DAB committees 


were created to replace the 126 boards 
and councils that had carved out their 
own niches in the bureaucratic 
cliff. 

With respect to acquisition, the 
VCJCS was specifically given respon¬ 
sibility for: (1) bringing a strategic 
viewpoint to acquisition decision¬ 
making, (2) working for reduced 
duplication of capabilities and increas¬ 
ed interoperability based on opera¬ 
tional lessons learned, (3) chairing the 
JROC, (4) vice chairing the DAB, (5) 
representing the OJCS on the Nuclear 
Weapons Council (6) acting as CJCS 
spokesman on PPBS matters and as his 
representative to the Defense 
Resources Board (DRB); and, (7) act¬ 
ing as CINC representative on acquisi¬ 
tion and requirements matters. 

Strategic Viewpoint 

What we acquire to equip the com¬ 
ponents must be consistent with who, 
where, what, when, and how we plan 
to use our military forces to achieve 
national strategic goals. This seems to 
be a first principle of defense acquisi¬ 
tion, but there has been a growing feel¬ 
ing that our arsenal was in charge of 
our strategy. 

The VCJCS has been charged with 
ensuring that the horse is, in fact, in 
front of the cart. This is especially im¬ 
portant as we deal with the concept of 
fiscally constrained strategies. We can¬ 
not afford to be spending scarce 
resources on development programs 
which may do more to reinforce the in¬ 
stitutional identity of the developer 
than contribute to the national 
defense. 

Reduced Duplication and 
Increased Interoperability 

Frankly, jointness is tough. It's 
tough on the battlefield, and it's tough 
in the program office. But just as 
frankly, there is only one war—no 
land war, no air war, no sea war- 
just one war. Every unnecessarily 
duplicated capability reduces the 
resources available to deal with other 
problems. Every system which is insuf¬ 
ficiently interoperable is a boon to the 


DISCLAIMER- This article represents the views of the author and does not necessarih reflect the official views of the Organization 
of the Joint Chiefs of Staff or the Department of Defense. The article is based on the author's presentations to the DSMC Executive 
Refresher and Executive Management Courses, and has been reviewed by the appropriate Joint Staff offices. 
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bad guys, and a bull's-eye on our own 
instep. The Joint Staff will be ag¬ 
gressively supporting the Vice Chair¬ 
man's efforts to make sure that these 
forms of jointness are achieved. An en¬ 
tire Joint Staff Directorate, the Direc¬ 
tor for Operational Plans and Inter¬ 
operability, J-7, has been established 
to seek after, define and monitor the 
implementation of means to improve 
our combatant forces interoperability. 
The J-7 also provides staff support for 
the JROC. The Director for Com¬ 
mand, Control, and Communications 
Systems, J-6, has been the focal point 
for interoperability for C3 systems and 
will continue to be. 

Joint Requirements 
Oversight Council 

As JROC Chairman, the VCJCS has 
had the Council rechartered to move 
its deliberations further "upstream" 
toward Milestone "0" to impact con¬ 
cepts as they develop rather than mere¬ 
ly adjust programs as they are ex¬ 
ecuted. Intuitively, it is better to have 
jointness considered early and applied 
prudently than it is to have it forced 
into a program after it has been struc¬ 
tured. Two mechanisms are necessary 
to make the JROC effective in this role. 
One deals with Service development 
activities and the other with CINC 
capability requirements. 

The annual Joint Potential Review 
and Designation Process is the 
mechanism dealing with Service 
development activities and produces 
the Joint Potential Designation List 
(JPDL). The intent of the process is to 
stimulate communication among the 
Services. All new Service and 
specified/unified command require¬ 
ments documents, all research and 
development (R&D) programs ap¬ 
proved for initiation and included in 
the Service Program Objective 
Memorandums, and all programs fac¬ 
ing a Milestone I or II review in the 
next fiscal year are subject to review. 
All four Services must review, com¬ 
ment on, and assign Joint Potential 
Designators (JPD) to all candidate pro¬ 
grams. The JPDs are required for all 
programs from acquisition category I 
through IV and are as follows: 
—Joint, a potential for joint R&D pro¬ 
gram management and/or joint pro¬ 
curement exists. 


—Interoperating, joint program 
management is inappropriate, but a 
potential for joint operation or joint 
system interface exists. 
—Independent, no potential for other 
Service use or joint systems 
development. 

The mechanism for dealing with 
CINC capability requirements is under 
development. It has much in common 
with the U.S. Army Concept Based Re¬ 
quirements System. The intent is to 


We cannot 

afford to be 
spending scarce 
resources on 
development pro¬ 
grams which may 
do more to rein¬ 
force the institu¬ 
tional identity of 
the developer 
than contribute 
to the national 
defense. 


provide a method for developing con¬ 
ceptual approaches to solving CINC 
problems before component respon¬ 
sibilities are defined rather than hav¬ 
ing component involvement dictate the 
concept. 

Both of these mechanisms will pro¬ 
vide information central to DAB 
deliberations, and the Vice Chairman, 


JCS, is uniquely positioned to facilitate 
JROC-DAB communication and inter¬ 
action. The ultimate goal is for the 
Joint Staff to have a well-defined posi¬ 
tion on the requirements for, and 
relative merits of, every DAB level 
program. 

Defense Acquisition Board 

With the VCJCS, as the DAB Vice 
Chairman, articulating Joint Staff posi¬ 
tions which balance "jointness" and 
marginal return against prudent 
duplication of capability, the DAB car. 
properly focus on how we buy what 
we buy; there are other forums for 
deciding what we buy. This means that 
the Joint Staff will be focused on keep¬ 
ing non-military influences from im¬ 
properly changing the goals of a 
development program from what was 
intended when the requirement was 
validated. 

The OJCS is not there to side with 
OSD against a Service, nor with a Ser¬ 
vice against OSD. The OJCS can be 
expected to take an independent posi¬ 
tion on each program, and that posi¬ 
tion will be rooted in linking strategy 
to plans, plans to requirements, and re¬ 
quirements to programs. 

This means that the Joint Staff will 
participate fully in DAB Committee 
activities. The current ten committees 
were created to replace an anarchic ap¬ 
proach to program oversight. Each 
committee has a Joint Staff Flag or 
General Officer member. Because the 
Joint Staff has limited acquisition pro¬ 
fessional resources these committee 
responsibilities have been "matrixed 
out." 

(It is useful to remember when deal¬ 
ing with the 1,400 member Joint Staff 
that fewer than 140 have an occasional 
contact with acquisition issues, some 
14 deal with acquisition daily, and 6 
or so are acquisition management 
professionals.) 

The major workload in support of 
DAB activities is handled by the Direc¬ 
tor for Force Structure, Resource, and 
Assessment Directorate, J-8, and the 
J-7. The J-7 oversees programs up to 
the draft Systems Concept Paper, and 
J-8 takes over from there through 
system retirement. The J-7 is respon¬ 
sible for seeing that the tie between 
plans and requirements is bound tight- 














ly, and the J-8 makes sure that the re¬ 
quirement is adhered to as the program 
proceeds. Some of the other Joint Staff 
Directors have similar responsibilities 
for certain classes of systems, such as 
Chemical Weapons and Nuclear 
Warheads (J-5). 

The VCJCS role in the Nuclear 
Weapons Council (NWC) is similar to 
his role in the DAB, but his support for 
NWC activities comes almost ex¬ 
clusively from the Director for 
Strategic Plans and Policy, J-5. 

Planning, Programming, and 
Budgeting 

Once the DAB has determined that 
a given program's risks and acquisition 
strategy are consistent with program 
goals, the Defense Resources Board 
(DRB) determines whether, and at 
what level, the program will be 
funded. Though not often thought of 
in these terms, it is actually the DRB 
which provides Milestone "0" ap¬ 
proval, not the DAB. As the CJCS 
representative to the DRB, the Vice 
Chairman will be working to ensure 
that DRB decisions underwrite stable 
programs, and that adjustments to ex¬ 
isting programs do not balance the 
books at the expense of our strategic 
goals. 

To make sure that he is in a position 
to do this effectively, the VCJCS is 
directing the Joint Staff evolution of an 


improved capability to conduct cost- 
trade analysis in support of develop¬ 
ing fiscally constrained strategies. 

CINC Spokesman on Acquisition 
And Requirements 

The VCJCS will be sponsoring 
CINC requirements for considera¬ 
tion/validation when normal processes 
aren't appropriate or aren't equal to the 
task. As noted, most of this function 
will be assumed by the JROC but the 
final responsibility will rest with the 
Vice Chairman. 

This means the VCJCS will 
direct/request Joint Staff/Service 
assistance to CINCs in developing Mis¬ 
sion Need Statements and presenting 
Mission Area Analysis concerns. 
Ultimately, however, the VCJCS must 
balance CINC theater needs agains f the 
tendency to proliferate theater specific 
weapon systems. 

Program Manager 

The OJCS primary concern is the 
growing time required to move from 
promising technologies to deployed 
systems. From our perspective, pro¬ 
gram instability is key; it begins when 
requirements are defined poorly or 
become moving targets as the threat 
changes over long development 
periods. It is aggravated by a decision 
process which allows bureaucratic 
"gate-keepers" to delay programs for 


narrow parochial concerns. Finally, 
our budget process can leave even our 
best program managers focusing their 
skills on reducing budget uncertainty 
through program advocacy rather than 
reducing cost, schedule, and perfor¬ 
mance risk through sound program 
management. 

The Defense Reorganization Act and 
NSDD 219 have provided the OJCS 
with the means to help reduce these 
sources of instability so that program 
managers can concentrate on the inter¬ 
nal elements of their programs a little 
more and the externals a little less. 
During the coming years, program 
managers can expect to see more sister- 
Service uniforms in the meeting room, 
more joint funding initiatives, and less 
tolerance for the discovery of "dis¬ 
jointed efforts." 


Lieutenant Colonel Wain is a Weapon 
Systems Program Evaluator for the Joint 
Staff. As such, he prepares Chairman and 
Vice Chairman, Joint Chiefs of Staff, to 
participate in SECDEF Performance 
Reviews, Defense Resources Board, Defense 
Acquisition Board, and Joint Requirements 
Oversight Council activities on USAF ma¬ 
jor system programs. He is the Joint Staff 
acquisition policy analyst. 


GANTT MEDAL IS AWARDED 


William S. Lee III has received the 
Henry Lawrence Gantt Medal from the 
American Society of Mechanical 
Engineers and the American Manage¬ 
ment Association. The medal is 
awarded annually for distinguished 
achievement in management as a ser¬ 
vice to the community. It was 
established in 1929 and memorializes 
Henry Lawrence Gantt, engineering 
manager, industrial leader, and 
humanitarian. 

Mr. Lee, chairman and chief ex¬ 
ecutive officer of Duke Power Com¬ 
pany, headquartered in Charlotte, 
N.C., was recognized before 1,500 in 
Chicago "for the forward looking 
policies that have made his organiza¬ 
tion a leader in providing opportunities 
for minorities; for outstanding leader¬ 


ship examples in the development of 
safe, dependable power; for his con¬ 
tinuing efforts to protect both the en¬ 
vironment and the communities in 
which his organization operates; for 
his unflagging efforts to aid education 
through organizing scholarship pro¬ 
grams within his organization and sup¬ 
port councils to help meeting the needs 
of public schools, as well as his own 
personal efforts to aid students; for his 
overall management skills which have 
made his organization a leader in the 
field." 

The American Society of Mechan¬ 
ical Engineers members on the Gantt 
Medal Board include David D. Acker 
of DSMC, Frank S. Locke, Nathan H. 
Hurt, Jr., William K. McAleer, and 
Robert Simmons. 


AFIT SEEKING 
FACULTY 

The Air Force Institute of Tech¬ 
nology (AFIT) teaches systems acquisi¬ 
tion/management courses to Air Force 
and DOD personnel in programs 
leading to masters degrees and in¬ 
programs of continuing professional 
education. It is seeking well-qualified 
individuals interested in teaching in 
those programs. Expressions of interest 
should be addressed to: AFIT/LS (Dr. 
W. Mauer), Wright-Patterson ArB, 
Ohio 45433-6583. Questions should be 
addressed to Professor William Dean, 
(513) 255-3355 or AV 785-3355. 
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PMC 88-1 GRADUATES 
HEAR DR. COSTELLO 


T he Honorable Robert B. 

Costello, Under Secretary of 
Defense (Acquisition), in his remarks 
to the Program Management Course 
88-1 graduating class at the Defense 
Systems Management College in May, 
said "this graduation is significant 
because it is the first of three DSMC 
will have during 1988 under the new 
accelerated PMC structure, replacing 
the former throughput of two classes 
annually. 

"I recognize the tremendous effort 
all of you on the faculty and staff have 
put into making this new process 
work. I know the overlap in classes 
tended to be confusing at first, but it 
all came together. 

"Things have changed at DSMC, 
most prominently the mission. Now, 
DSMC is responsible for all acquisition 
education, rather than just program 
management education as before... .To 
each of you here today, continue to 
work not only for the good of the col¬ 
lege, but for improving everything we 
do in defense acquisition." 

Dr. Costello said graduates are 
military and civilian officials and are 
among tomorrow's top government 
and industry leaders. Each has been 
recognized as having high potential to 
be the best. "A lot is riding on your 
shoulders," he said, "and the directions 
and actions our country takes in the 
future will depend upon judgments and 
decisions you and counterparts will 
make. It is an awesome responsibility 
and must be handled judiciously. If 
there is one single point I wish to drive 
home today, it is that Department of 


Program Manager 


Defense people must make changes in 
mindset, form and substance—the 
total picture—in order to improve the 
way we do business. What we in the 
Department of Defense have set out to 
change will be done gradually, not 
overnight, but it must be done. 

"Changes must be initiated at the 
top, encouraged at the top, im¬ 
plemented at the top, and led at the 
top. Top management must be com¬ 
mitted to change, and then follow 
through to accomplishment. 

Meet Challenges 

"Whether you will be in manage¬ 
ment or staff positions, you must sup¬ 
port efforts of those above you to 
make the necessary changes, and ful¬ 
ly execute them down the line. As you 
advance in your careers, these and 
other challenges await you. Meet them 
head-on. 

"Do not be satisfied with, or fearful 
of, established institutions as mono¬ 
lithic intimidators. Believe that the 
Department of Defense way of doing 
business can be done better. Have 
courage of your conviction to put in¬ 
to practice only that which is best for 
our fighting forces, and for national 
security. Discard the superfluous. 


"Your leadership ability will be 
tested severely in coming years. Tight 
financial constrictions...will be with us 
for the forseeable future....Many of 
you will work on the program objec¬ 
tives memorandum and budget at the 
Pentagon and at headquarter levels 
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....You will wrestle with probable 
budget cuts, program delays, stretch¬ 
outs and perturbations. Your man¬ 
agerial skill to get the most for 
available dollars will bear on your job 
performance but, most importantly, 
will greatly affect the end product— 
whether it be weapon platforms, 
strategic systems, communications, 
size of forces, or numbers of 
people. 

"The management and leadership 
principles you have honed at the 
Defense Systems Management College 
will prepare you well for years to 
come, and help you make it through 
exciting, turbulent and, occasionally, 
stressful times." 

Dr. Costello said that it stands to 
reason that management and leader¬ 
ship principles improving the acquisi¬ 
tion process apply across-the-board, in 
every facet of government. He told 
graduates they will inherit many deci¬ 
sions of the past; some will bind, 
others will not. "Your contribution to 
government, and the payoff of your 
DSMC education will be to use proper 
initiative, common sense, and judg¬ 
ment to make the ultimate hard 
decisions—and make them right," he 
said. 

Dr. Costello concluded his remarks 
by saying that "it takes leadership to 
convince people and established in¬ 
stitutional processes that changes will 
make everyone’s job easier, and pro¬ 
vide the best and least expensive 
systems for the field. 


These are excerpts from Dr. Costelb’s 
graduation address. His ten agenda items 
for improving defense acquisition were de¬ 
tailed in the May-June 1988 issue of 

Program Manager. 
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DISPELLING MYTHS OF 
TEST AND EVALUATION 

Dr. H. Steven Kimmel 

Assistant Deputy Director 
Defense Research and Engineering 
(Test and Evaluation) 


T est and Evaluation is 
recognized as a key 
element of the weapon 
system process. By long¬ 
standing practice and direc¬ 
tive, weapon system test and 
evaluation is divided into 
two principal categories— 

Development Test and Eval¬ 
uation (DT&E) and Opera¬ 
tional Test and Evaluation 
(OT&E). As defined by the 
governing directive (DoDD 
5000.3, "Test and Evalua¬ 
tion"), DT&E is conducted 
throughout various phases 
of the acquisition process to ensure the acquisition and 
fielding of an effective and supportable system by assisting 
in the engineering design and development and verifying at¬ 
tainment of technical performance specifications, objectives 
and supportability. The OT&E is the field test, under 
realistic conditions and by typical users of the weapon 
system (or element thereof) to determine its suitability and 
effectiveness. 

While DT&E emphasizes engineering design and technical 
performance, its ultimate goal, like that of OT&E, is to en¬ 
sure the acquisition and fielding of weapon systems that are 
effective and supportable under combat conditions. One 
should not exoect DT&E, by itself, to be sufficient to fully 
ensure effective, supportable combat operation. Key 
elements of realistic testing are reserved to OT&E; e.g., 
operation by typical military users in as realistic a represen¬ 
tative field condition as possible against threat representative 
hostile forces. Nonetheless, it is clear that the utility of DT&E 
as an acquisition tool is increased when Development Test 


(DT) results can serve as a 
reliable predictor of Opera¬ 
tional Test (OT) perfor¬ 
mance. 

Figure 1 presents a 
simplified model of the 
weapon system acquisition 
process to assist in examin¬ 
ing the relationship between 
DT&E and OT&E. The 
model suggests that test and 
evaluation is a continuum of 
activities interwoven with 
the acquisition process. In 
reality, the maturing DT&E 
and initial phases of OT&E (IOT&E) events do not fit into 
rigid or discrete compartments; both are involved with 
broad, system-level concerns. This relationship is a matter 
of ongoing interest and often the cause of confusion and 
misunderstanding. In addition, recent enactments by the 
Congress have drawn attention to the Office, Secretary of 
Defense (OSD) T&E communities' management, execution 
and actions. The result of this certainly needs ti he well 
understood by those engaged in the acquisition of weapon 
systems. Accordingly, the reader is invited to take and 
evaluate the test on the next page. 

The intent of this test is not to provide pass/fail criteria 
for the reader, but to clarify ongoing relationships between 
DT&E and OT&E communities. More precisely, the entire 
T&E community is being challenged to support the evolv¬ 
ing acquisition community goals and objectives. Using viable 
test programs and the commitment to objective assessments 
are the essential means to achieve results recently expressed 
by the Army Chief of Staff General Carl E. Vuono: 
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FIGURE 1. MODEL OF WEAPON SYSTEM ACQUISITION PROCESS 



Test and Evaluation is very 
important because putting less 
than adequate weaponry into the 
hands of our soldiers is a price 
that we all cannot afford. 


Fifteen Questions 

1. The OSD Test and Evaluation en¬ 
vironment is divided and fragmented. 

T or F 

2. Test and Evaluation assessments 

go unheeded. T or F 

3. Weapon system acquisition time 
has increased due to the demands/re¬ 
quirements for more testing. T or F 

4. Existing T&E policies of DOD and 
military departments are inconsistent 
in philosophy and approach. T or F 

5. Development Test (DT) and Initial 

Operation Test (IOT) are incompati¬ 
ble in purpose. T or F 

Program Manager 


6. Test and Evaluation Master Plans 

(TEMPs) are needed solely to comply 
with DoD Directive 5000.3, "Test and 
Evaluation"; i.e., to fulfill a process re¬ 
quirement. T or F 

7. Until recently, software T&E has 

placed emphasis more on "T" than "E" 
and has thus endorsed the Design-Test- 
Redesign-Retest philosophy of 
development. T or F 

8. Live Fire Testing is solely an Army 
test program brought about by the 
Bradley Live Fire Test (LFT) effort. 

T or F 

9. Non-development item (NDI) is an 

acquisition strategy that eliminates the 
requirement for Test and Evaluation 
master planning. T or F 

10. A balanced structured system test 

plan treats hardware, software and the 
user as equals. T or F 

11. Preplanned product improvements 

(P3I) or evolutionary acquisition (EA) 
strategies (yes, there is a difference) 
minimize the conflict between DT and 
OT. T or F 
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12. Concurrent acquisition strategies 
typically reduce schedule risk at the ex¬ 
pense of informed decisions. T or F 

13. Operational suitability and opera¬ 

tional effectiveness are graduation 
characteristics validated as a conse¬ 
quence of OT. T or F 

14. Nuclear hardness and survivabili¬ 
ty (NH&S) objectives are to be 
achieved in DT and confirmed in OT. 

T or F 

15. Weapon system (platform) com¬ 
bat survivability is a function of 
susceptibility and vulnerability. T or F 


Answers to Questions with Sup¬ 
porting Rationale 

1. The OSD Test and Evaluation en¬ 
vironment is divided and fragmented. 

False. That is, while the OSD T&E 
function is fragmented organizational¬ 
ly, its purpose is not. Clearly, the 
Director of Operational Test and Eval¬ 
uation (DOT&E) is responsible for 
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providing OT policy, while the Depu¬ 
ty Director of Defense Research and 
Engineering (Test and Evaluation), 
DDDRE(T&E), is responsible for DT 
policy generation. However, both en¬ 
tities agree that: 

—'Test planning" must begin early 
—Early test results are essential to sup¬ 
port design to production decisions 
—The DT objectives must posture a 
system OT 

—The OT must be viewed as a gradua¬ 
tion type of event rather than a final 
exam. 

A single OSD organization would 
not alter this ideology. Two organiza¬ 
tions permit focused attention upon a 
myriad of related matters such as test 
facilities (instrumentation and ranges), 
test targets, threat surrogates and 
simulators, live fire testing, joint tests, 
and foreign weapon evaluation. Most 
importantly, it allows independent, 
objective weapon systems assessments 
from a development perspective to en¬ 
sure that engineering thresholds have 
been attained and from an operational 
perspective to verify the suitability and 
effectiveness is confirmed before pro¬ 
ceeding beyond low-rate initial 
production. 

2. Test and Evaluation assessments 
go unheeded. 

Fi' “. The T&E assessments are an 
integral influence in the defense ac¬ 
quisition decision-making process. 
Routine vehicles such as weekly "quick 
look" T&E correspondence and more 
detailed status reports and memoran¬ 
da are frequently generated to ensure 
the Secretary of Defense and Under 
Secretary of Defense (Acquisition) are 
appraised of acquisition T&E matters. 
Lastly, the T&E assessments provided 
to the Defense Acquisition Board often 
form the foundation of the resultant 
Acquisition Decision Memorandum 
that directs the Service Acquisition Ex¬ 
ecutive during the forthcoming 
months of program execution, 

3. Weapon system acquisition time 
has increased due to the demands/re¬ 
quirements for more testing. 

False. During 1987, as a result of 
congressional interest, a thorough and 
comprehensive review of the weapon 


system acquisition process judged that 
a vigorous, well-planned T&E effort is 
time-efficient and a wise investment. 
The conclusion from the evaluation of 
testing time is that T&E is not a 
reasonable target for time reduction in 
pursuit of a speedier acquisition pro¬ 
cess, and that a cutback in T&E could 
actually lead to a longer and costlier 
acquisition process. 

4. Existing T&E policies of DOD and 
military departments are inconsistent 
in philosophy and approach. 

False. The DcDD 5000.3, "Test 
and Evaluation," sets forth the broad 
philosphical basis for T&E and iden¬ 
tifies specific responsibilitites and 
methods for all T&E actions in the 
Department of Defense. It highlights 
the purpose of testing, the relationship 
between DT & OT as currently de¬ 
fined, specific military service respon¬ 
sibilities, and T&E planning and ex¬ 
ecution requirements. 

The military services, in turn, have 
each prepared regulations that imple¬ 
ment the guidance contained in DoDD 
5000.3 for their respective organiza¬ 
tions. In general, the military services 
are in concert with the fundamentals 
of the DOD T&E guidance, although 
there are differences in terminology 
and approach. For the most part, this 
is a consequence of the different 
weapon system classes and operating 
environments with which each mili¬ 
tary service must deal. 

5. Development Test (DT) and Initial 
Operational Test (IOT) are incompati¬ 
ble in purpose. 

False. The T&E is an integral part 
of the acquisition process, interacting 
with other program functions that sup¬ 
port the development of a product 
design and helping to determine the 
operational effectiveness and suitabili¬ 
ty of a weapon system. Testing (devel¬ 
opment and operational) must begin 
early and be done continuously rather 
than viewing it as a "final exam." Ac¬ 
cordingly, the relationship between 
DT and IOT is separate, yet com¬ 
plementary. For example, an early 
operational assessment is often based 
upon the DT data. In the end, they 
contribute to a healthy T&E environ¬ 
ment which, in turn, helps the DOD 
in its decision process. The DT con¬ 


tributes to the design portion of the 
process, whereas OT establishes the in¬ 
formation base for recommending 
when a program is ready to proceed 
past a low-rate initial production. 

6. Test and Evaluation Master Plans 
(TEMPs) are needed solely to comply 
with DoDD 5000.3, 'Test and Evalua¬ 
tion"; i.e., to fulfill a process 
requirement. 

False. The purpose of a TEMP is 
to: 

—Identify the scope of planned testing 
—Delineate acceptable evaluation 
criteria 

—Foster sound program management 
by which to execute a rational, logical 
course of action. 

The concept of a TEMP is de¬ 
tailed in DoD Manual 5000.3-M-l, 
Test and Evaluation Master Plan 
Guidelines, and supports the premise 
that the document should be viewed as 
a tool to ensure program management 
success. The TEMP should serve as a 
roadmap rather than a detailed test 
plan. Accordingly, program officials 
should use it as a vehicle to confirm, 
via test results, the progress of a 
maturing set of events. This progress 
is examined continuously during the 
development period and is necessary 
to facilitate the certification that the 
developing system under test is ready 
for OT. Utilizing evolutionary 
thresholds that span the phases of ad¬ 
vanced and engineering development 
right through the certification for OT 
readiness and leading to initial opera¬ 
tional capability can become bona fide 
checkpoints to support acquisition 
management decisions. 

Hence, the TEMP is and should be 
viewed as a living document, accom¬ 
modating annual updates and revi¬ 
sions. Meanwhile, the TEMP must 
track with the approved requirements 
and Decision Coordination/System 
Papers to ensure that the objective 
system, once fully fielded, satisfies the 
military user's required operational 
needs (final set of evolutionary 
thresholds). 

Accordingly, military service sub¬ 
mitted TEMPs receive a thorough and 
comprehensive OSD-wide review to 
ensure that programmatic and tech- 
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nical viewpoints, concerns and details 
are adequately addressed. 

7. Until recently, software T&E has 
placed its emphasis more on "T" than 
"E" and has endorsed the Design-Test- 
Redesign-Retest philosophy of 
development. 

True. The design-test-redesign ap¬ 
proach has been observed to be costly 
(see note below) and inefficient for 
hardware and software development 
efforts. Therefore, the soon-to-be 
published DoD Manual 5000.3-M-3 

"Software Test and Evaluation 
Manual" will advocate the following: 

1. DoD-STD-2167, "Defense 
Systems Software Development," 
which established a tri-service ap¬ 
proach for designing and building in 
software quality vice testing in soft¬ 
ware quality. Quality is one measure 
of software maturity. 

2. Development of mathematical 
means of determining software correct¬ 
ness analytically during design and 
prior to code development, leading the 
way to error reduced software being 
developed in a "clean-room" environ¬ 
ment. This is still in its infancy. 

3. Support the DOD implemen¬ 
tation of Ada for use in all weapon 
systems. This approach fosters the ear¬ 
ly detection and prevention of require¬ 
ment and design errors prior to soft¬ 
ware coding. 

4. Improving management visi¬ 
bility, testing, and assessment tools 
which promote management attention, 
early detection, and correction of soft¬ 
ware problem areas. 

NOTE: Evidence produced by soft¬ 
ware studies indicates that, on the 
average, approximately 40-60 percent 
of DOD software development dollars 
are spent on software test-related 
activities. 

8. Live Fire Testing is solely an Army 
test program brought about by the 
Bradley Live Fire Test (LFT) effort. 

False. Vulnerability of the Bradley 
Fighting Vehicle was initially quan¬ 
tified by the OSD sponsored, joint 
Live Fire Test effort. In FY 87, the 
Congress established a DOD-wide 
Live Fire Test mandate to be overseen 
by OSD and executed by each military 


service. The enabling legislation states 
that LFT shall be executed sufficiently 
early in the development phase of the 
system or program to allow any design 
deficiency demonstrated by the vulner¬ 
ability testing to be corrected in the 
design of the system, munition or 
missile before proceeding beyond low- 
rate initial production. 

The TEMP is the umbrella document 
to record the scope of a system, muni¬ 
tion or missile's LFT. The supporting 
Detailed Live Fire Test and Evaluation 
Plan contains the subsystem and com¬ 
ponent or full-up testing needed to 
assess system vulnerability or lethali¬ 
ty. Such a full-up test may be waived 
by the Secretary of Defense prior to 
entering Full-Scale Development and 
provided that the Secretary certifies to 
the Congress that LFT would be un¬ 
reasonably expensive and impractical. 

9. Non-development item (NDI) is an 
acquisition strategy that eliminates the 
requirement for Test and Evaluation 
master planning. 

False. The use of NDI is based 
upon the results of a market surveil¬ 
lance and analysis performed early in 
the system life cycle. Typically, the 
analysis determines the feasibility of 
satisfying a military deficiency or need 
by utilizing commercial off-the-shelf 
products. These products may be used 
either directly, or ruggedized for 
military environments, or integrated 
into existing or evolving system 
design(s). In any case, the degree of 
testing will be commensurate with the 
degree of integration and/or modifica¬ 
tion required. It will be inversely pro¬ 
portional to the depth of contractor 
data appropriate for evaluating mili¬ 
tary operational suitability and 
effectiveness. 

As described in the forthcoming 
DoD 5000.3-M-5, "Procedures Man¬ 
ual-Improving Test and Evaluation Ef¬ 
fectiveness in Support of the Major 
Weapon Systems Decision Process," 
the testing of an NDI must not impede 
the objectives of streamlining the ac¬ 
quisition process. Rather, it must be 
accomplished in an orderly, objective 
fashion consistent with the approved 
TEMP. 


10. A balanced structured system test 
plan treats hardware, software and the 
user as equals. 

True. A balanced test plan is 
necessary to ensure that system and 
mission objectives will be supported by 
maturing hardware and software. The 
test plan must address hardware (in¬ 
cluding mission critical computer 
resources as well as front-end sensors), 
software (firmware as well as resident 
and data flow induced), user elements 
and the integration of all of these 
elements. System-level testing should 
be designed and conducted to demon¬ 
strate the contribution of hardware, 
software and people to the quantifica¬ 
tion of reliability, availability, and 
maintainability parameters. 

11. Preplanned product improvements 
(P3I) or evolutionary acquisition (EA) 
strategies (yes, there is a difference) 
minimize the conflict between DT and 
OT. 

True. The P3I and EA contain a 
modular building block concept to 
enable the integration of progressive 
hardware and software design en¬ 
hancements capable of meeting 
futuristic mission objectives by the 
phasing-in of upgradable intrinsic 
elements. 

The enhancements result in cost 
avoidance in such areas as obsolete 
system software, planned delivery of 
new technology and the tailoring of 
operational characteristics to increase 
the utility of available, employable 
technology. 

Hence, at the system level, the adap¬ 
tation of P3I or EA tends to minimize 
DT and OT conflicts by accommo¬ 
dating design changes during a spec¬ 
trum of time as evidenced by specific 
development objectives and OT expec¬ 
tations. With a balanced test program 
consisting of integration, interopera¬ 
bility, and compatibility testing, the 
modular concept can provide the road¬ 
map to reduce test and acquisition 
conflicts. 

12. Concurrent acquisition strategies 
typically reduce schedule risk at the ex¬ 
pense of informed decisions. 

False. In the usual context, concur¬ 
rency means either t v e simultaneous 
DT/OT or the more common interpre- 


Program Manager 


11 


July-August 1988 











tation of simultaneous DT/OT and 
production. 

With concurrent DT/OT the pro¬ 
gram office and contractor are faced 
with the dilemma of trying to collect 
developmental data as quickly as 
possible while providing and support¬ 
ing a system for the independent 
operational tester to verify user re¬ 
quirements. What often happens is the 
curtailment or retarding of develop¬ 
mental data taking to ensure com¬ 
pliance with a particular operational 
performance threshold, thus impeding 
overall development and progress 
toward certification for dedicated 
system level OT. Some concurrency in 
DT/OT is beneficial but only to the ex¬ 
tent that quick-look OT evaluations 
are used to assist the developer and 
help refine system specifications. 

D -rimental concurrency of DT/OT 
occurs, for example, when the pro¬ 
gram office/contractor team candidly 
admits their attention has been 
diverted from developmental to opera¬ 
tional testing. Typically, this means 
that the preference to collect data (i.e., 
to fire several telemetry equipped 
missiles) followed by the opportunity 
to correct engineering deficiencies has 
been set aside in deference to the 
necessity to rehearse IOT&E to 
minimize the possibility of any embar¬ 
rassing operational occurrences. Thus, 
the attainment and confirmation of 
development objectives as verified 
through flight test become mitigated. 

13. Operational suitability and opera¬ 
tional effectiveness are graduation 
characteristics validated as a conse¬ 
quence of OT. 

True. The latter phase of OT (i.e., 
OTII), is the place to prove operational 
effectiveness. Whereas some of opera¬ 
tional suitability can be assessed from 
the results of DT, DoD 5000.3 states 
that operational suitability under 
realistic conditions is to be validated 
during OT. 


14. Nuclear hardness and survivabili¬ 
ty (NH&S) objectives are to be 
achieved in DT and confirmed in OT. 

True. The DoDI 4245.4, "Acquisi¬ 
tion of Nuclear-Survivable Systems," 
states that "NH&S objectives are (to 
be) achieved during DT and OT&E." 

In addition, DoDD 5000.3, "Test 
and Evaluation," states that "DOT&E 
is responsible for (confirming), in coor¬ 
dination with the Assistant to the 
Secretary of Defense for Atomic 
Energy, ATSD(AE), that OT&E con¬ 
firms NH&S as intended." 

"DDDRE (T&E) is responsible for 
confirming, with advice from the 
ATSD(AE), that NH&S objectives are 
achieved during DT&E." 

As we are aware, testing to confirm 
NH&S must depend upon a combina¬ 
tion of tests, simulations and analysis. 
Like the non-nuclear survivability 
area, severity of the mission degrada¬ 
tion will be extrapolated in terms of the 
ability of the threat operating environ¬ 
ment to exceed the system design capa¬ 
bilities, thereby affecting system sur¬ 
vivability. The degree to which the 
system survives will directly contribute 
to mission abort, mission degradation, 
nuisance or no response. In the end, 
the system's design (development ob¬ 
jective) must be robust to lie some¬ 
where between impervious to the ex¬ 
pected threat (i.e., no vulnerability), 
to built-in recovery (observable degra¬ 
dation) and the acceptance of partial 
(but militarily acceptable) operational 
suitability and effectiveness. 

15. Weapon system (platform) com¬ 
bat survivability is a function of 
susceptibility and vulnerability. 

True. Survivability is "the 
capability of a system to avoid and/or 
withstand a man-made hostile environ¬ 
ment" without suffering abortive im¬ 
pairment of its ability to accomplish its 
designated mission. 


One element of survivability must 
be expressed in terms of the intrinsic 
system ability to use whatever possi¬ 
ble natural phenomena, countermeas¬ 
ures or tactics to avoid weapons 
(ballistic projectiles, guided missiles, 
exploding warheads, and other ele¬ 
ments that make up the hostile en¬ 
vironment). That is, the measure can 
be expressed by P h , the probability 
that the system may be hit by a 
damage-causing mechanism, and is 
referred to as the susceptibility of the 
system. 

The inability of a system to with¬ 
stand the damage caused by the hostile 
environment is referred to as the 
vulnerability of the system to the 
damage mechanisms. Vulnerability 
can be measured by P k/h , the condi¬ 
tional probability that the system will 
be killed or degraded given a hit by a 
damage mechanism. 

The extent that a system's effec¬ 
tiveness may be reduced by the hostile 
environment is measured by the prob¬ 
ability the system is killed P k or 
degraded P d . The probability of kill of 
the system is the product of the prob¬ 
ability of hit (the susceptibility), P k , 
and the conditional probability of kill 
given a hit (the vulnerability), P k/h , 
that is: 

Probability of Kill = Susceptibility 
* Vulnerability P. = P h * P k/h is ap¬ 
propriate for conditions of permanent 
damage; i.e., high explosive induced, 
and Probability of Degradation = 
Susceptibility * Vulnerability P d = P h 
*P d /h * s appropriate for conditions of 
temporary damage; i.e., electro¬ 
magnetic induced. 

Thus, the survivability of the system 
can be expressed as a capability to en¬ 
dure in a hostile environment as 
measured by the probability of sur¬ 
vival P s . Its relationship to P k is given 
by P s = 1 - P k . 
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C/SCSC LESSONS LEARNED 
THEORETICAL FRAMEWORK 

Dr. Anthony Webster 


TT5 

111 rogram managers 

JL. for the government, 
and defense contractors have 
come a long way since early 
implementation of the 
Arthur D. Little Cost/ 

Schedule Control System 
Criteria (C/SCSC) in the 
early 1967-68 time frame. 

Currently, there is less ap¬ 
prehension of the criteria, 
and the cost to implement 
appears to be reasonable in 
most cases. Contractor and 
government personnel con¬ 
tinue to learn more and bet¬ 
ter ways to use C/SCSC and analyze performance measure¬ 
ment data. Many observations have been made and many 
lessons have been learned. 

In 1975, a C/SCSC Implementation—Lessons Learned- 
study was conducted by L. S. Morrella and J. B. Holeman 
as a project during the Defense Systems Management Col¬ 
lege PMC 75-1. That paper documented the "corporate 
memory" of C/SCSC experiences and lessons learned for 
the 7-year span, 1968-75. 

My paper builds on earlier research, attemping to com¬ 
pare deficiencies and lessons learned identified in 1975 with 
current observations of deficiencies in the discipline (12 years 
later). 

Lessons learned as observed in the 1975 study are pro¬ 
vided in Table 1. 


Other areas of co: cem 
were program instability, 
schedule variance expressed 
in dollars, accounting for 
material at point of usage, 
level of detail involved, cost 
associated with implement¬ 
ing and inflexible interpreta¬ 
tion of criteria. 

The Lessons Learned (Fig¬ 
ure 3, pp. 20-22) reflects per¬ 
spectives taken from seven 
vantage points as a/an: 

—Contractor performance 

measurement instructor working with user population 
—Observer of feedback from C/SCSC validation reports 
—Participant in C/SCSC focal points and review director 
meetings 

—Participant in NSIA Management Subcommittee meetings 
—Consultant providing C/SCSC support to government 
agencies 

—Member of the Performance Management Association 
—Participating team member on demonstration and subse¬ 
quent application reviews. 

Background 

Observation #1. A recent Air Force study reveals that of 
86 reviews conducted from 1984-87, 70.5 percent were 
satisfactory, 11.4 percent marginal and 18.2 percent un¬ 
satisfactory. The deficiency distribution by criteria categories 
for those reviews are shown in Table 2. 
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TABLE 1. LESSONS LEARNED IN 1975 STUDY 

c/scsc 

SUBJECT 

REMEDY 

Retrofit applications 
(not cost effective, disruptive) 

Retrofit applications should be 
avoided. 

Premature demonstration 
reviews (not cost effective, 
disruptive) 

Realistic schedules must be 
developed. Meaningful readiness 
assessments. Contractor should 
not drag out process. 

Lack of program management 
office involvement 

Encourage Department of Defense 
(DOD) program managers to have 
office well represented on the 
review team. (Big dividends in 
terms of understanding and 
visibility). 

Lack of headquarters 
guidance 

Provide adequate and standard 
guidance to all review team 
members. Make it the responsi¬ 
bility of the review director to 
implement a standard interpreta¬ 
tion of the criteria and be well- 
trained in the C/SCSC discipline. 


The review director should be pro¬ 
vided by the major military service 
headquarters. 

Lack of plant representative 
involvement. 

Enforce DOD requirements that con¬ 
tract administrators and auditors 
conduct continuing monitoring and 
surveillance of contracts. The Cost 
Accounting Standards (CAS) and 
Defense Contract Auditing Agency 
(DCAA) should be involved in 
demonstration reviews. The 
Memorandums of Agreement 
(MOAs) should be established. 

Top management support 

“Units do well those things the Boss 
checks.” Top management on 
both sides must embrace and sup¬ 
port the requirement. 

Analysis and use of data 

Help contractor and government 
analysts to have a better apprecia¬ 
tion of data value and emphasize 
a greater understanding of data 
analysis, and utilization. 

Estimates at Completion (EACs) 
review team members 

Emphasize the need for meaningful 
and disciplined EACs. The EACs 
should be realistic and timely. 
Turnover must be minimized and 
adequately trained. 
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TABLE 2. CRITERIA CATEGORIES 


NUMBER C/SCSC CATEGORY 


II Planning and budgeting 


IV Analysis 


Revisions and access to data 


TABLE 3. DEFICIENCIES 


FREQUENCY OF 
OCCURRENCE PERCENT 


Estimate of cost at completion 
Selection of earned value technique 
Level of effort application 
Subcontractor performance measurement 
Baseline establishment and control 
Variance analysis and reporting 
System discipline 
Material control 
Validity of computer runs 
Reconciliation of external reporting data 
with internal data 


II TABLE 4. PROBLEMS IMPLEMENTING C/SCSC 


PROBLEM CATEGORY 


System not required for years (not 
updated) 

Deviations from validated system 
Poor internal audits 


Implementation problem 


Training schedule traceability 
Application of earned value 
techniques 

Quality of variance analysis 


Government problem 


Undefinitized contracts 
Unrealistic target cost 
Poor surveillance 
Program office allows 
non-compliance 
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Other areas of concern by the Air 
Force were excessive data reporting re¬ 
quirements, delayed reporting of 
material and subcontractor cost 
variances and qualifications of 
C/SCSC Review Team members and 
Teeth in C/SCSC (remedies) for 
non-compliance. 


Observation #2. The Navy analyzed 
18 Validation Review Reports of con¬ 
tracts totaling $1.8 billion and iden¬ 
tified the 10 most frequently reported 
deficiencies that could affect validity 
of cost data. Table 3 shows the defi¬ 
ciencies and their frequency of occur¬ 
rences observed during the 18 reviews. 


Observation #3. In a recent presen¬ 
tation made by the Tri-Service Focal 
points, current problems implementing 
C/SCSC were identified and divided 
into three major categories—systems 
problems, implementation problems 
and government problems. Table 4 
shows the information provided in that 
presentation. 


Observation #4. In an interview 
with a C/SCSC management consul¬ 
tant who has supported the govern¬ 
ment, I asked him to provide a list of 
major deficiencies in implementing 
C/SCSC based on his company's ex¬ 
periences. Data provided are shown in 
Table 5. 
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TABLE 5. MAJOR DEFICIENCIES IN 
IMPLEMENTING C/SCSC 


PROBLEM CATEGORY 

REASONS 


Planning 

—Incorrect identification of cost 
accounts 

—Weak relationship between the 



Work Breakdown Structure (WBS) 
and statement of work 



—Inadequate development of the 

WBS 

—Poor definition of work 
—Inadequate integration of budget, 



schedule, and work authorization 



—Overallocation of budget 
—Forward planning at too high a 



level 


Control 

—Work status not based on 



work package progress 
—The Budgeted Cost for Work 



Performed (BCWP) technique not con- 

Observation #5. Feedback from 


sistent with resource application 

Contractor Performance Measurement 


—Performance measurement at 

Course students, C/SCSC Seminars I 


incorrect level 

have amended for the last 5 years and 


—Improper or late incorporation of 

speaking with major defense contrac- 


changes j 

tors provide a composite overview of 


—Invalid baseline maintenance 

perceived key problems in the im¬ 


system 

plementation of C/SCSC. 

They are: 


—Material costs not applied properly 

—Interfacing with existing systems 

Accounting system 

—Accounting periods do not match 

—Establishing and maintaining WBS 
vs. cost accounts 


customer or subcontractor account¬ 
ing periods 

—The 12 vs. 13 accounting periods 

—Developing code fields to integrate 


during the year 

cost/schedules 


—Reports prepared after accounting 

—Establishing objective eamed-value 


month closing 

techniques 


—Inadequate follow-up to reduce 

—Keeping the baseline up-to-date 


errors 

—No recording of labor and material 

— Reporting against unrealistic 


transfers between departments and 

baselines 


projects 

—Accounting for materials and multi¬ 
function equipment 

Data analysis 

—Variance analysis system not 

—Handling subcontractors (especially 

maintained 

in research and development) 


—Performance inadequately 

—Using Management Reserve as a 


monitored 

management tool. 


—Corrective actions not followed up 
—Estimates at completion not 



updated 


System maintenance 

—System design too complex 
—Documentation not accurate 
—Training not done continuously 
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TABLE 6. OPPORTUNITIES FOR IMPROVEMENT 

PROBLEM CATEGORY 

REASONS 

Baseline change 

The DOD change order pro¬ 
cess does not permit 
work to start prior to 
receipt of authorization. 

Lack of timely accounta¬ 
bility in Performance 

Measurement Baseline 
(PMB). 

Review process 

Lack of consistency in the 
criteria interpretation be¬ 
tween review teams. 

Preconceived changes 
directed prior to the 
scheduled review. 

C/SCSC application 

Lack of flexibility in its 
implementation during 
the research and develop¬ 
ment phase. 

Variance analysis 

Formal documentation of 
analysis is time con¬ 
suming. 

Some reporting is repetitious, extra 
paperwork. 

Thresholds are too low. 

CPR reporting 

The levels for reporting 
requirements are too low. 

WBS 

The MIL-STD 881-A should be 
a guide, not a mandate. 

TABLE 7. MAJOR AREAS FOR IMPROVEMENT 

PROBLEM CATEGORY 

REASONS 

C/SCSC review process 

Team member qualifications (not 
well qualified). 

Turnover rate is high. 

Career path problems. 

C/SCSC application 

Lack of consistency in criteria. 

interpretation 

Time allowed for implementation. 

Lack of incentives for a contractor 
to work C/SCSC. 

Lack or understanding of C/SCSC 
capabilities and limitations by 
both government and industry. 

Cost performance reports 

Excess reporting. 

Timeliness of reports. 

Quality of variance analysis. 


Observation #6. Table 6 identifies 
areas some Defense contractors believe 
presented the greatest opportunity for 
improvement in the implementation of 
C/SCSC. 


Observation #7. The C/SCSC 
survey of Department of Defense and 
contractor managers Phase II was also 
a source considered for background in¬ 
formation. Phase II of referenced 
survey identified problems, whose 
resolution could lead to improvements 
in the implementation of C/SCSC. 
The major areas for improvement 
identified in this survey are provided 
in Table 7. 


APPROACH 

Analysis of Data 

The observations give background 
information to show a representation 
of major deficiencies/problems asso¬ 
ciated with implementing C/SCSC and 
Lessons Learned. They represent a 
cross-section of government and in¬ 
dustry program management sources 
throughout the country. This data 
covers from 1968 through 1987. The 
referenced data set includes: The 
C/SCSC Study, "Lessons Learned," 
1968-75; Air Force C/SCSC Review 
Results, 1987; Navy C/SCSC Review, 
1986; C/SCSC Tri-Service Presenta¬ 
tion, 1987; C/SCSC Management 
Consultant Interview, 1987; Feedback 
from CPMC student population and 
C/SCSC seminars, 1983-87; Consor¬ 
tium of Defense Contractors: C/SCSC 
Response, 1987; and Arthur D. Little 
C/SCSC Survey, 1986. 

The deficiencies identified through 
the data set were stratified into 27 
categories and put into a matrix. See 
Figure 1, C/SCSC Implementation 
Deficiency Matrix, to determine the 
frequency of occurrence among the 
eight sources used in the data set. The 
frequency distribution is shown in 
Figure 2. 
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[FIGURE 1. 

\ C/SCSC DEFICIENCY 
IMPLEMENTATION 
MATRIX 


SOURCE (DATA) 


m 



1. etrofit C/SCSC appl. 

2. Premature demonstrations 

3. Lack of PO involvement 

4. Lack of HQ guidance 

5. Plant representatives 

6. Top management support 

7. Analysis/use of data _ 

8. Estimate at completion 

9. Review team members 

10. Planning and budgeting 

11. Accounting _ 

12. Analysis (variance) _ 

13. Earned value _ 

14. Subcontractor _ 

15. Material control _ 

16. Reports _ 

17. Training _ 

18. Schedules _ 

19. Authorized undefinitized 

20. WBS _ 

21. Baseline _ 

22. Management reserve 

23. LOE application _ 

24. System discipline _ 

25. Revisions _ 

26. Work authorization _ 

27. Non-compliance 
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FIGURE 2. THE C/SCSC DEFICIENCY FREQUENCY DISTRIBUTION TABLE 


DEFICIENCY 

CATEGORY 

ORDER OF HIGHEST 
FREQUENCY '(See Note) 

BASIS FOR LESSONS 
LEARNED TABLE 

1. Analysis/use of data 

6 

VARIANCE ANALYSIS 

2. Analysis (variance) 

6 


3. Training 

6 

TRAINING 

4. Reports 

5 

REPORTS 

5. Earned value 

4 

EARNED VALUE 

6. Material control 

4 

MATERIAL CONTROL 

7. Review team members 

4 

REVIEW TEAM 

8. Estimate at completion 

3 

EAC 

9. Accounting 

3 


10. Subcontractor 

3 

SUBCONTRACTOR 

11. Baseline maintenance 

3 

BASELINE 

12. Work breakdown structure 

3 

WBS 

13. Systems discipline 

3 

SYSTEM DISCIPLINE 

14. Revisions 

3 

REVISIONS 

15. Lack of program office 

2 

TOP MGMT SUPPORT 

involvement 



16. Top management support 

2 


17. Plant representative 

2 


18. Planning and budgeting 

2 


19. Schedules 

2 

SCHEDULES 

20. Authorized undefinitized 

2 


21. C/SCSC non-compliance 

22. Lack of headquarters involvement 

23. Retrofit C/SCSC application 

24. Premature demonstrations 

25. Management reserve 

26. LOE application 

27. Work authorization 

2 

1 

1 

1 

1 

1 

1 

MANAGEMENT RESERVE 



'mmpi 


|P$| 


SfiSsK 


'NOTE: Frequency of occurrence is on attached C/SCSC Implementation 
Deficiency Matrix. 
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FIGURE 3. THE C/SCSC LESSONS LEARNED (1987) 

SUBJECT 

PROBLEM 

SOLUTION 

VARIANCE ANALYSIS 

. Inherent reaction time of variance data 
not sufficient to predict each individual 
problem. 

. Continue to stress observation 
of early warning. 


. Documenting superficial detail problems 
can be time-consuming and non-productive. 

. Set variance thresholds to maximize ad¬ 
verse trend analysis. 


. Lack of timely and meaningful data/variance 
analysis. 

. Stress not only variance thresholds. 

. Quantify the variance. 

. Describe the problem and its impact on 
cost, schedule and other elements. 

. Recommendation/corrective actions. i 

. Emphasize program manager involvement. 


. Too stringent threshold requirements 
(Retards proper management and analysis). 

. Thresholds should vary depending upon size 
and sensitivity of reporting element. 1 


. Value of Cost Performance Report (CPR) 
schedule variance may not indicate a real 
schedule problem. 

. Don’t ignore the schedule variance; it is a 
preliminary indicator of cost problems and a 
useful trending tool. 

. Variance expressed in dollars.. .conversion to 
time can be a useful tool. 

TRAINING AND IMPLE¬ 
MENTATION OF C/SCSC 

. Lack of consistency in the interpretation 
of the criteria. 

. Provide team members with work instruc¬ 
tions; flexible, functional procedures. 


. Lack of training. 

. Require formal training for team members. 



. Smal 1 classes and on-the-job training. 

. Require subteam establishment with at 
least one qualified member. 


. Lack of C/SCSC emphasis at top levels of 
management. 

. Develop and distribute a C/SCSC 
newsletter. 


. Lack of understanding of C/SCSC 
capabilities and limitations. (Government 
and Industry) 

. Special courses for commanders, chief 
executive officers (CEOs) and vice 
presidents. 

. Involve top-level management in the 
reviews. 


. Lack of consistency in C/SCSC application. 

. Annual meetings of C/SCSC review direc¬ 
tors and team chiefs. 

. Involve the participation of the Office of the 
Secretary of Defense, training institutions, 
and cross fertilization of the military 
services on review teams. 


. Preconceived changes directed prior to the 
scheduled review. 

. Educate the contractor about the appeals 
process. 

VARIANCE ANALYSIS 

. Analysis and use of the data. 

. Improvise ways to help contractor and 
government analyst to better appreciate the 
value of the data and emphasize a greater 
understanding of how to analyze and utilize 
the data. 

REPORTS 

. Excessive data/reporting requirements. 

. Validate proposed requirements and review 
current requirements. 

. Educate personnel on optimal data needs. 


. Delayed reporting of materials and sub¬ 
contract cost variances. 

, Require tracking in Format 5 of performance 
against a material cost baseline as material 
commitments and subcontract awards are 
made. 
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FIGURE 3. THE C/SCSC LESSONS LEARNED (1987) (Continued) 

SUBJECT 

PROBLEM 

SOLUTION 

REPORTS (cont’d.) 


. Require monthly adjustments of EAC based 
on reported material cost variances and ac¬ 
tual commitment. 


. Lack of reconciliation of external reports 
with the CPR. 

. In Format 1, require reporting by cost 
element (labor, material and other direct). 

. Reconcile CPR with Contractor Cost Data 
Report (CCDR). (Aids in cost rejections.) 


. Timeliness of the reports, and quality of. 
the variance analysis. 

. Service focal points should assure that the 
contractor is aware of the need for timely 
and quality data for the reports to yield their 
maximum utility. Use the contractual vehicle 
to assure compliance. 

. Incremental submittal of CPR Formats. 

. Implement the use of electronic data 
transmission when possible. 

EARNED VALUE 

. Insufficient measurement techniques. 

(Too much level of effort (LOE)) 

. Avoid use of LOE. 

. Use all available quantitative, objective 
measurement techniques. 

. It is better to revert to subjective assess¬ 
ment techniques rather than allow higher 
percentage of LOE. 

. Maximize ratio of measurable vs. LOE to 
prevent distortion of performance 
measurement. 


. Mix of LOE within discrete cost accounts. 

. Mixture is allowable within certain 
guidelines and constraints. 

MATERIAL CONTROL 

. Lack of material accountability and status 
on contracts. 

. Ensure that the system provide visibility 
into the entire material status of contract. 



a. Should be identified in WBS element. 

b. People responsible for material. 

c. Earned value for material. 

ESTIMATE AT COMPLETION 
(EAC) 

Adequacy of EAC projection difficult to 
assess: Frequency, application of automa¬ 
tion, application of unreliable overhead 
rates. 

. Evaluate EACs by trend analysis. 

. Require periodic detail EAC preparation. 

. Intensify use of “Cum-to-date” cost 
variances. 


. Cost growth tends to occur in steps. 
Reluctance of manager to admit growth. 
Time constant between problem identity 
and cost impact. 

. Supplement manager estimates with 
parametric techniques. 

. Allow for program uncertainties when 
establishing management reserve (MR). 

SUBCONTRACTOR 

. Lack of subcontractor visibility. 

(Sometimes the subcontractor does not ap¬ 
pear in the prime contract work breakdown 
structure (CWBS). 

. Management Control System flowdown 
required. 

. Must be addressed in solicitation and con¬ 
tracts clause. 

. Prime tasked to perform validation and 
surveillance to subcontractor. 


. Subcontractor reporting non-standard. 

. Should be compatible with the prime, via CPR, 
C/SSR with required procedures for taking 
BCWP. 


. Subcontractor EACs. 

. EACs must be monitored for the 
subcontractor. 
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FIGURE 3. THE C/SCSC LESSONS LEARNED (1987) (Continued) 




SUBJECT 


PROBLEM 


SOLUTION 


BASELINE 


SYSTEM DISCIPLINE 


REVISIONS 


SCHEDULES 


MANAGEMENT RESERVE 


. Baseline establishment and control. 

(Weak relationship between WBS and state¬ 
ment of work (SOW). 

. Baseline not representative of all work on 
the contract. 


. Manufacturing cost accounts 
—Inappropriate WBS levels. 

—Arbitrary time spans. 

. Multiple budgeting and tracking techniques. 
. Government/contractor system under¬ 
standing. 

. System not in use for years. 


. Authorizations 

—Approval signatures dated after work 
starts, or no dates. 

—Many log entries dated the week before 
the arrival of the review team. 

. Authorized undefinitized work. 

(Not incorporating effort into the PMB can 
jeopardize contract performance.) 

. Timeliness of internal changes. 


. Lack of schedule traceability (horizontal) 
detailing interdependencies: 

—Problem (intermediate level schedules) 
no horizontal trace between key events 
and major milestones. 

. Lack of sufficient predecessor-successor 
identification. 

. Reserve source must not be speculative 
(e g., labor rate windfalls). 

. Criteria for application must be definitive and 
restrictive. 


. Internal trace of work from contract line item 
numbers (CLINs) to SOW, to WBS cells/WBS 
dictionary. 

. Baseline should reflect status of authorized 
unpriced work. 

. Subcontractors must appear in the baseline. 

. Establish cost accounts at meaningful cost 
account levels. 

. Allov, cost accounts to represent total task. 

. Adapt system to internal methods. 

Establish working rapport with government 
surveillance monitors. 

. Establish internal focal points. 

. Periodic self-audits. 

. Use system to manage all programs. 

. All authorizations which include tasK, budget, 
and schedule. 

. Maintain current logs for changes, MR and 
undistributed budget. 

Obtain Procuring Contracting Officer (PCO) 
Authorization to proceed and incorporate 
proposal/NTE value as an authorized 
undefinitized change. 

. Changes require immediate incorporation 
into the baseline. 

. Contractual requirement for submission of 
network schedules. 

. Expansion of intermediate schedules to 
identify interdependencies. 

. Require quarterly submission of summary 
network. 


. Segregate reserves for labor rate fluctuation 
from technical or program contingencies. 

. Reserves should be controlled by higher 
management. 


Conclusion 

This article tells the C/SCSC story 
and provides corporate memory of 
C/SCSC problems/solutions from 
1968-87. The data were gathered from 
a cross-section of government and in¬ 
dustry program management sources 
throughout the country. 

Overall, this data reflects a history 
of steady progress toward successful 
implementation of the discipline. 
Much of the data indicates there has 
been a transition from problems of im¬ 
plementation to analysis and use of 
data. Training and analysis and use of 
data stood out as priority among the 
problems currently associated with im¬ 


plementing C/SCSC. The data also in¬ 
dicate that the discipline still lacks the 
kind of top management support that 
is conducive to future success in im¬ 
plementing C/SCSC. This suggests 
that government and industry top 
management should be more involved 
in the review process and perhaps a 
top-level management C/SCSC course 
should be established. 

Nonetheless, in collecting this data 
it became obvious there was a lack of 
consistency in how C/SCSC data has 
been collected over the years. It has 
ranged from that of subjective surveys 
to frequency of deficiency occurrences. 


Based on this observation, I recom¬ 
mend establishing a set of standard 
parameters for data collection for Air 
Force, Army, Navy, NSIA, and other 
C/SCSC data sources. In time, this 
collection of a standard data set will 
provide a greater opportunity to 
analyze, compare and extract pertinent 
information from our C/SCSC data 
bases. 


Dr. Webster is a Professor of financial 
management at the Defense Systems 
Management College. 
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Measuring & Managing 
o Quality 

OipRODUCTIVITY 
_ ON_ 


T his fall, a document will be issued describing the best 
practices in quality and productivity measurement 
and management on defense systems. Those of us concerned 
with preparing this document hope it will gain widespread 
acceptance in the defense and aerospace industry because 
current practices need to be, and can be, improved. 


£)EFENSE PROGRAMS 


David Dr Acker 



The importance of measurement was recognized by Lord 
Kelvin in 1883 when he stated: "I often say that when you 
can measure what you are speaking about and express it in 
numbers, you know something about it; but when you can¬ 
not express it in numbers, your knowledge is of a meagre 
and unsatisfactory kind; it may be the beginning of 
knowledge, but you have scarcely, in your thoughts, 
advanced to the stage of science, whatever the matter may 
be." 1 

We should not deal with quality and productivity as 
though they are intangibles. Rather, we should deal with 
them in numbers; otherwise, results will be of "a meagre 
and unsatisfactory kind." 

The "Creed for Productivity Improvement," as stated by 
J. L. Riggs and G. H. Felix in their textbook on productiv¬ 
ity, underlines an important point 1 would like to make. 2 
According to these writers. 

To improve productivity you must manage. 

To manage effectively, you must control. 

To control consistently, you must teasure. 

To measure validly, you must define. 

To define precisely, you must quantify. 

Quality and productivity measurement and management, 
once considered to be arduous tasks have been improved, 
but can be improved further. Recognizing this, some 
knowledgeable people working in university-based produc¬ 
tivity centers and leading defense industrial firms in the 
United States have been engaged in a five-year, five-phased, 
management-oriented quality and productivity project, 
funded by the Department of Defense, to see how and where 
improvements might be made. Findings have been reviewed 
by an industry advisory group, 3 and the Office of Secretary 
of Defense and military service representatives at selected 
checkpoints throughout the project. Results will be published 
in the fall. 

I participated in the project from the beginning and served 
as the contracting officer’s representative for three of the 
five years. This landmark project addressed important issues 
in quality and productivity measurement and management 
on defense programs. Before publication the guide, setting 
forth project results, is being reviewed and evaluated by 
executives and managers of industrial firms, and the Office 
of the Secretary of Defense (OSD) and Military Department 
representatives. The guide will incorporate recommended 
changes. 

Let's review project phases leading to preparation and 
publication. 
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Summary of Efforts By Project 
Phase 

Phase I was conducted under leader¬ 
ship of Monte Norton and Wayne 
Zabel at the Army Procurement 
Research Office (APRO), Fort Lee, 
Va., with encouragement from 
Dr. Richard A. Stimson, then Direc¬ 
tor of Industrial Productivity, Office 
of the Under Secretary of Defense 
(Research and Engineering). 4 The 
effort involved a survey of then- 
current productivity measurement and 
evaluation techniques used in the 
defense industry. Some definitions 
relative to contractor productivity, 
acceptable to the defense industry, 
were developed. 

The Phase I study revealed several 
things. 

—As a performance evaluation factor, 
industrial contractors ranked produc¬ 
tivity below profitability, effec¬ 
tiveness, quality, and efficiency. 
—Problems encountered in industrial 
firms' attempts to measure productiv¬ 
ity are usually caused by difficulties 
experienced in identifying and quan¬ 
tifying inputs and outputs. 
—Visibility of production costs varies 
widely in industry. In addition, 
visibility of indirect costs is becoming 
more important because of shifts in 
cost drivers. 

—Tracking the impact of improve¬ 
ment interventions is difficult, espe¬ 
cially in indirect cost areas. 
—Investments are made more often for 
competitive and technological reasons 
than for reducing costs on current 
contracts. 

—Industry must learn to integrate con¬ 
ventional and new approaches to per¬ 
formance measurement and evalua¬ 
tion, applying them when and where 
appropriate. 5 


TABLE 1. MODELS AND METHODOLOGIES FOR 
MEASURING PRODUCTIVITY 

MODEL/ 

METHODOLOGY 

DESCRIPTION 

Total Factor 

Productivity Measurement 
Model (TFPMM) 

The TFPMM is a dynamic, aggregated, indexed, and 
computerized approach to measuring productivity. 

TFPMM can be utilized to measure productivity 
changes in labor, capital, materials, energy and 
data/information. It measures the corresponding effect 
each one has on profitability. 

The TFPMM is based on the premise that profitability 
is a function of productivity and price recovery; that is, 
an organizational system can generate profit growth 
from productivity improvement and/or from price 
recovery. Productivity relates to quantities of output 
and quantities of inputs, while price recovery relates 
to prices of output and costs of inputs. 

Cost Definition 

Methodology 

(CDEF) 

The Price Waterhouse CDEF methodology is an 
approach for preparing cost baseline data in support 
of factory modernization efforts for both commercial 
factories and factories participating in the Department 
of Defense Industrial Modernization Incentives Pro¬ 
gram (IMIP). CDEF utilizes a top-down analysis tech¬ 
nique which facilitates the identification of appropriate 
performance and cost measurement criteria, selection 
of improvement opportunities (through capital invest¬ 
ment and/or efficiency improvements), and economic 
justification of identified investments. 

Discounted Cash Flow/ 
Shared Savings Approach 
(DCF/SSA) 

The DCF/SSA model consists of (1) a time-phased 
model of cash receipts (or savings) and cash 
disbursements over a finite planning horizon, with a 
portion of the receipts being shared savings, and (2) a 
calculated internal rate of return measure of economic 
effectiveness. 

LTV Integrated 

Productivity Measurement 
Model 

The LTV model incorporates elements of the TFPMM, 
CDEF, and DCF/SSA methodologies for measuring 
productivity. 


Phase II was conducted by the 
Oklahoma Productivity Center at 
Oklahoma State University, and the 
Maryland Center for Productivity and 
Quality of Working Life at the Univer¬ 
sity of Maryland, under contract to the 
Air Force Business Research Manage¬ 
ment Center (AFBRMC). Colonel 
Ronald Deep, USAF, Director of 
AFBRMC, was the contracting officer's 
representative. This phase focused on 
identifying and describing available 


productivity measurement techniques 
and developing a taxonomy of produc¬ 
tivity measurement theories and 
techniques. 

Phase II concluded three things. 
—Knowledge of specific productivity 
measurement techniques is not 
widespread. 

—State-of-the-art techniques require 
substantial efforts to implement. 
—Some macro-measurement and sur- 
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rogate techniques may be adequate for 
a specific manager's needs. 6 

Phase III was conducted under con¬ 
tract to the Defense Systems Manage¬ 
ment College (DSMC) by the Virginia 
Productivity Center (VPC) at Virginia 
Polytechnic Institute and State Univer¬ 
sity. Dr. D. Scott Sink, Director of 
VPC, was principal investigator. This 
phase focused on evaluating produc¬ 
tivity measurement techniques iden¬ 
tified in Phase II; i.e.. Total Factor Pro¬ 
ductivity Measurement Model 
(TFPMM), Price Waterhouse Cost 
Definition Methodology (CDEF), Dis¬ 
counted Cash Flow/Shared Savings 
Approach (DCF/SSA), and LTV Inte¬ 
grated Productivity Measurement 
Model. Models and methodologies for 
measuring productivity are presented 
in Table 1. Four subcontractors plus 
principals directly involved in Phase III 
were the Maryland Center for Produc¬ 
tivity and Quality of Working Life, 
Dr. Thomas C. Tuttle; LTV Aircraft 
Products Group, Shoni Dhir; Price 
Waterhouse, Betty Thayer and 
William Muir; Westinghouse Defense 
Group, Richard Engwall. 

Phase 111 concluded several 
things. 

—Individually, none of the first three 
models evaluated could accomplish all 
objectives desired by industrial firms 
or government. 

—A methodology incorporating a 
variety of measurement and evaluation 
models is required. Models, when inte¬ 
grated (as was done by LTV), con¬ 
stitute a potentially satisfactory 
methodology to accomplish industry 
and government objectives; that is, 
industrial firms want to improve com¬ 
petitiveness and profits. On the other 
hand, the government wants to lower 
acquisition costs and improve product 
quality. 

—Variances in organizations, manage¬ 
ment styles, pressures, priorities, 
perceived problems and opportunities, 
and skilled/competent management 
personnel make it difficult to translate 
and transfer models and methodolo¬ 
gies from one company to 
another. 7 

Phase IV was conduced by the 
Virginia Productivity Center and the 
subcontractors identified previously. 


under contract to the Defense Systems 
Management College with funding 
provided by the Office of the Secretary 
of Defense. This phase consisted of a 
field test of the TFPMM, CDEF, and 
DCF/SSA and the LTV Integrated Pro¬ 
ductivity Measurement Models; com¬ 
pletion of the productivity manage¬ 
ment methodology designed in Phase 
III ; design and development of a draft 
guide to communicate philosophies, 
principles, tools, and techniques of 
quality and productivity management 
(generally referred to as performance 
management); and development of 
detailed plans for evaluation 
workshops to be held in Phase 

V S9 

Phase V is conducted by the Virginia 
Productivity Center and subcontrac¬ 
tors identified previously under con¬ 
tract to the Defense Systems Manage¬ 
ment College with Office of the 
Secretary of Defense funding. The goal 
is to complete and publish the guide 
drafted in Phase IV. Feedback from an 
evaluation workshop during this phase 
should enhance the quality of the guide 
before release. 

The Guide 

The guide will describe good perfor¬ 
mance measurement and management 
practices used in the defense industry 
in America today. Dr. Sink and Paul 
Rossler, the project manager, and 
members of the project team are con¬ 
vinced the measurement and manage¬ 
ment practices described in the new 
guide will help each organization 
achieve and maintain a competitive 
position in the defense system acquisi¬ 
tion business in the 1990s and beyond. 

The performance management pro¬ 
cess, which is easy to conceptualize 
(see Figure 1) is not easy to place in 
operation. If a manager understands 
the basic process, but fails to obtain a 
proper understanding and agreement 
within the organization on the process 
and specific tools needed to effectively 
implement it, that organization will 
not reach its goal. 

The process shown in Figure 1 may 
appear complex at first glance; 
however, it is a simple depiction of 
how many excellently run companies 
in America measure and manage qual- 


f Jl he Guide 

will describe 
good performance 
measurement and 
management 
practices used in the 
defense industry in 
America today. 


ity and pi Juctivity. It depicts the 
organization as an open system with 
various feedback loops from the envi¬ 
ronment, and it highlights interrela¬ 
tionships among components of a per¬ 
formance management process. 

Probably the best place to start 
describing the model is in the middle 
with the box titled "Organizational 
System." This box represents the 
system being managed. It could be a 
company, division, plant, or depart¬ 
ment. The system has upstream 
systems (internal and external suppliers 
and vendors) which provide inputs in 
the forms of labor, material, capital, 
energy, and data/information. The 
system takes these inputs and converts 
them into outputs in the form of goods 
and services. Downstream systems 
(internal and external customers) react 
to these outputs, creating outcomes 
(customer satisfaction, readiness, 
profitability). 
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At the top of Figure 1 is the new 
competition the organization must 
respond to in order to compete in a 
global economy. This new competition 
and global economy influence com¬ 
pany business strategy and company 
visions of the organization of the 
future. The company assesses its pres¬ 
ent performance, and uses this data as 
a foundation for developing plans for 
performance improvement. Key per¬ 
formance indicators are identified to 
provide the company with feedback on 
its progress. 

Out of the performance improve¬ 
ment planning process comes specific 
performance improvement interven¬ 
tions, tactics and techniques. Note that 
these interventions happen at five 
quality checkpoints: (Ql) upstream 
systems; (Q2) inputs; (Q3) process; 
(Q4) outputs; and (Q5) downstream 
systems. After interventions to the 
system have been made, performance 
is measured, assessed, and analyzed at 
the five quality checkpoints to deter¬ 
mine whether the impact the interven¬ 
tions were to have made on the system 
actually occurred. Based on this data, 
an evaluation relative to the com¬ 
pany’s business strategy, environment 
(internal and external), vision, plans, 
and improvement interventions are 
made. Note that the process of evalua¬ 
tion is separate from the process of 
measurement. Measurement supports 
improvement and this, of course, is of 
primary interest. 

If a company has an effective, high- 
quality management process in the 
areas of planning, measuring and 
evaluating, and controlling and 
improving, it will achieve its vision of 
the future, and its desired outcomes. 

Before I comment on the guide in 
more detail, it is important that you 
have a basic understanding of the 
quality and productivity terminology 
used. 


The definitions listed below 
apply. 

Effectiveness. The degree to which 
a system accomplishes what it sets out 
to accomplish. 10 

Efficiency. A comparison among re¬ 
sources expected to be consumed to 
accomplish a specific goal, task, or 
objective and resources actually con¬ 
sumed to reach that goal, accomplish 
the task, or meet the objective. 11 

Innovation. Creative process of 
adapting products, services, and pro¬ 
cesses to changing internal and exter¬ 
nal pressures, demands, changes, and 
needs. It might be called the process of 
maintaining fitness for use from the 
customer's viewpoint. 


Performance (Quality and Produc¬ 
tivity) Management. Process which in¬ 
cludes planning, measurement and 
evaluation, control, and improvement. 
Figure 1 depicts the general flow of the 
performance management process. 

Productivity. Relationship of the 
quantity of output goods and/or ser¬ 
vices obtained from an organizational 
system for some period of time to the 
quantity of input resources consumed 
by that organizational system for the 
same period of time, quality 
considered. 

Productivity Improvement. Result 
of managing and intervening in key 
transformation processes. Productivity 
improvement occurs when any of the 
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following conditions are made to 
exist: 

—Output increases and input 
decreases 

—Output increases and input remains 
constant 

—Output increases and input increases 
but at a lower rate 
—Output remains constant and input 
decreases 

—Output decreases and input 
decreases but at a more rapid rate. 

Productivity Measurement. 12 Selec¬ 
tion of physical, temporal, and/or 
perceptual measures for input variables 
and output variables, and the develop¬ 
ment of a ratio of output measure(s) 
to input measure(s). A good produc¬ 
tivity measurement system will in¬ 
dicate to management when there is a 
need to plan and act, and provide clues 
where in the system to intervene. 

Profitability. Ability to yield advan¬ 
tageous results. It is a measure, or set 
of measures, of the relationship among 
financial resources and their uses, such 
as return on assets and return on 
investments. 


Quality. Degree to which a system 
conforms to requirements, specifica¬ 
tions, and expectations. To be man¬ 
aged effectively, quality must be de¬ 
fined relative to five quality check¬ 
points, namely: 

Ql: Selection and management of 
upstream systems (internal and exter¬ 
nal suppliers and vendors) 

Q2: In-coming quality control 

Q3: In-process quality management 

Q4: Out-going quality control 

Q5: Proactive and reactive 
assurance of customer satisfaction. 

These checkpoints were highlighted in 
Figure 1. 

Quality of Work Life. Efforts to in¬ 
crease performance through improve¬ 
ment in psychosocial rather than 
technical and structural systems. 

Total Quality Management. Man¬ 
agement system including all five 
quality checkpoints. (See “quality" 
definition.) 


Performance of an organization is a 
function of many of the criteria 
already defined. Conceptually, these 
criteria are interrelated in the manner 
shown in Figure 2. 

Desired Outcomes for the Guide 
Those playing parts in developing 
the guide hope it will do the 
following: 

—Communicate fundamentals of per¬ 
formance management methodology 
—Establish role and importance of 
productivity measurement and 
management 

—Move people in government and in¬ 
dustry away from a narrow view of 
productivity measurement and 
management 

—Demonstrate that measurement 
leads to improvement 
—Demonstrate that measurement sup¬ 
ports incentive methodology. 


FIGURE 2. THE CASUAL RELATIONSHIPS AMONG 
PERFORMANCE CRITERIA 
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Design Criteria for the Guide 


The design criteria, agreed to by the 
team in the fall of 1986 and revised in 
1987, are shown below. 


Concept of Guide 
The guide will set forth state-of-the- 
art practices based upon the 
knowledge, wisdom, and experience 
gained during the past ten years or so. 
It will reflect an appropriate blend of 
rigor and relevance, theory, technique, 
and practical applications. It will pre¬ 
sent practices in use today in some of 
the excellently managed manufactur¬ 
ing firms in America. The guide will 
be organized around a generic model 
(Figure 1) for performance manage¬ 
ment. It will provide a logical flow of 
the performance management process. 


—Prepare a guide written from the 
contractor's perspective and responsive 
to contractor and government needs 


—Present a procedure for designing a 
grand strategy for quality and produc¬ 
tivity improvement 
—Explain role of measurement in per¬ 
formance improvement efforts, espe¬ 
cially the link between measurement 
and incentive methodologies 
—Identify and describe, in succinct 
form, govemment-to-contractor incen¬ 
tive methodologies 
—Explain the Total-Factor Productiv¬ 
ity Measurement Model, Price 
Waterhouse Cost Definition Method¬ 
ology, Discounted Cash Flow/Shared 
Savings Approach, LTV Integrated 
Productivity Measurement Model, and 
other surrogate measurement tools and 
techniques 

—Document a technique that can be 
used to evaluate existing and proposed 
productivity measurement and man¬ 
agement systems 

—Provide a balanced view of the per¬ 
formance management process 
—Discuss philosophy for making per¬ 
formance management and improve¬ 
ment a way of life 
—Test, before formal publication, the 
guide credibility to government and 
contractor personnel. 


Xhc 


Lhe Guide 
will reflect an 
appropriate 
blend of rigor 
and relevance, 
theory, technique, 
and practical 
applications. 


Chapter 1. The Challenger. What 
the Defense Contractor of the Present 
Faces. This chapter concerns the 
tremendous stress that our global, 
dynamic economy is creating on 
American organizations. It explains 
that increased competition is forcing 
management teams to reexamine basic 
practices. It states how we do business 
must be scrutinized and modified. 
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Chapter 2. Visions. What the De¬ 
fense Contractor of the Future Must 
Look Like. Shared, well-communi¬ 
cated visions of the future are critical 
to achieving desired outcomes in the 
defense industry. Thus, the perfor¬ 
mance management process logically 
begins with a focus on visions, guiding 
principles, missions, and outcomes. 


Chapter 3. The Responses. Strategy 
and Plans for Continuous Quality and 
Productivity Improvement. This 
chapter provides an overview of the 
performance management process. It 
explains the process and shows how 
the process can be a "roadmap" for an 
organization’s performance manage¬ 
ment efforts. This chapter provides a 
detailed outline of what follows in the 
guide. This chapter discusses how to 
create plans for quality and productiv¬ 
ity improvement. It also focuses on a 
critical and oftentimes troublesome ele¬ 
ment of performance management, the 
planning process. 


Chapter 4. Measurement. Theory 
and Approaches: How to Build Qual¬ 
ity and Productivity Measurement 
Systems. This chapter presents the 
theory of measurement and the 
approach for building measurement 
systems. Critically important philoso¬ 
phy and foundation concepts that must 
guide measurement efforts are pro¬ 
vided. The distinction between 
measurement systems for control and 
improvement are discussed. 



tip 


Chapter 5. Measurement Tech¬ 
niques. How to Measure Quality and 
Productivity. This chapter explains the 
measurement techniques. Various 
models are presented for consideration. 



Chapter 6. Improvement. Strategies 
to Make Quality and Productivity 
Improvement Happen. This chapter 
explains how to improve quality and 
productivity. It provides a structural 
overview of the state-of-the-art. Many 
unique techniques are presented in a 
step-by-step, flow-process fashion. 




July-August 1988 






Techniques include recent techniques 
used in performance management 
efforts; e.g., govemment-to-contractor 
incentive methodologies (industrial 
modernization incentives program), 
contractor-to-employee incentive pro¬ 
grams, total quality management and 
participative management. 

Chapter 7. Maintaining Excellence. 
How to Make Quality and Productiv¬ 
ity a Way of Life. The final chapter is 
devoted to maintaining excellence in 
performance management. It's a tough 
task to sustain a culture supporting 
constant improvement and innova¬ 
tion. The reader should be challenged 
to think about the organization of the 
future. Questions like these will be 
raised: How does an organization 
function using concepts and techniques 
presented in this document? How does 
one keep moving toward a vision 
while simultaneously and continually 
updating the vision? 

Conclusion 

Measuring and managing quality 
and productivity at the close of the 
1980s, and into the 1990s, will con¬ 
tinue to be complex. This is partly the 
result of the complex interrelationship 
continuing to exist among government 
entities, competitors, the economy, 
technological advances, and the uncer¬ 
tain environment. Therefore, imple¬ 
menting even the simplest concept or 
technique presented here will be a 
challenge for any organization, indus¬ 
trial firm or government entity. 

The need for a guide is evident. The 
guide presents in a clear and succinct 
fashion a structured approach to per¬ 
formance management based on good 
practices being used successfully in 
1988. 

Although implementation of models 
and methodologies set forth in the 
guide will not ensure success, organi¬ 
zations using them properly will be 
more apt to succeed in performance 
management than those who do not. 
We should recognize that quality and 


productivity are not always what they 
seem to be on defense system pro¬ 
grams. Realities of a manager's expec¬ 
tations call to mind these lines from 
Act II of Gilbert and Sullivan's HMS 
Pinafore, which I had the pleasure to 
see again recently: 

Things are seldom what they seem, 

Skim milk masquerades as cream 


Storks turn out to be but logs, 


Bulls are but inflated frogs. 

Gild the farthing if you will. 

Yet it is a farthing still. 

Many factors influence a manager to 
gild the quality and productivity far¬ 
thing. The successful manager will be 
able to show it is possible to do more 
with less while using techniques not 
always understood by others and, 
sometimes, without an incentive for 
meaningful improvement. 
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and Westinghouse, 
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A. Stimson from the federal govern¬ 
ment in 1986, Douglas Reeves (now 
retired), Gordon A. Frank, and Kurt 
Greene, all from the Office of the 
Secretary of Defense, have kept a 
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Research Office (APRO) Number 
83-01, subject: "Contractor Productiv¬ 
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6. Report of the Air Force Business 
Research Management Center 
(AFBRMC) Number BRMC-83-5071, 
subject: "Development of a Taxonomy 
of Productivity Measurement Theories 
and Techniques," February 1984. 
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tivity Center (VPC), subject: "The 
Study of Productivity Measurement 
and Incentive Methodology (Phase Ill- 
Paper Test)," 3 volumes, March 1986. 

8. Report by the Virginia Produc¬ 
tivity Center, subject: "The Study of 
Productivity Measurement and Incen¬ 
tive Methodology (Phase IV-Field 
Test)," March 1987. 

9. Masters of Change: A Joint 
Defense-Industry Guide to Quality and 
Productivity (Draft). 

10. Peter Drucker defines efficiency 
as "doing things right" and effec¬ 
tiveness as "doing the right things." 

11. Ibid. 

12. According to Peter Drucker, 
"...In turbulent times, the first task of 
management is to make sure of the in¬ 
stitution's capacity for survival, to 
make sure of its structural strength and 
soundness, of its capacity to survive a 
blow, to adapt to sudden change, and 
to avail itself of new opportunities.... 
Without productivity objectives, a 
business does not have direction. 
Without productivity measurement, it 
does not have control." 


Mr. Acker is a Professor of management 
at the Defense Systems Management 
College. 
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PROTOTYPES 

Cabin Brown 


L ately, the term "prototype" has been used extensive¬ 
ly in articles, books and journals dealing with the 
acquisition of weapon systems and equipment. However, 
the term is rarely defined and the reader is left to determine 
his or her perception of which meaning the author intended. 
This article is intended to look at kinds of prototyping and, 
in the process, clear up some confusion. 

In the final report of the President's Blue Ribbon Com¬ 
mission on Defense Management (Packard Commission), 
competitive prototyping, either at the system or critical sub¬ 
system level, was recommended for all major weapon 
systems. In the advanced development phase of a program 
(concept demonstration/validation phase of a major 
system), this prototyping would involve an informal com¬ 
petition of ideas and technologies rather than a formal com¬ 
petition based on cost. Early operational testing, and 
developmental testing, should be performed to uncover 
operational and technical deficiencies before entering full- 
scale development (FSD). At this stage it is still relatively 
inexpensive to correct operational and technical deficiencies 
uncovered during testing of the prototypes. 


El arly operational testing and 
developmental testing should be 
performed to uncover operational 
and technical deficiencies before 
entering full-scale development. 


The Department of Defense Appropriations Act for fiscal 
1987 mandated that all major weapon systems entering ad¬ 
vanced development after September 30,1987, must include 
a competitive prototype strategy. The act requires that 
contracts be entered into with at least two contractors, and 
requires that systems and subsystems developed undergo 
comparative side-by-side tests. Contrary to the Packard 
Commission concept of "competitive prototyping," each 
contractor developing a prototype system or subsystem is 
required to submit cost estimates for full-scale development 
and production estimates, where practicable, before testing 
begins. When competitive strategy is not practicable, the 


Secretary of Defense must submit written notification to the 
Congress explaining that position. The latest revision of 
Department of Defense Directive 5000.1 reflects this 
mandate. 

The previous discussion reflects the meaning of "prototyp¬ 
ing" as defined in the DSMC Glossary of Defense Acquisi¬ 
tion Acronyms and Terms: "An original or model on which 
a later item is formed or based. Usually built during 
Demonstration/Validation and tested prior to Milestone II 
decision." This is normally referred to as an "engineering" 
or "experimental" prototype and is the meaning most often 
inferred by the term "prototype." 

A second kind of prototype is "preproduction" which is 
developed during full-scale development and represents the 
production configuration in all aspects of form, fit, and func¬ 
tion except that it has not been fully qualified in terms of 
environmental and operational testing. A preproduction 
prototype is produced for the purpose of final system/equip¬ 
ment/software test and evaluation before entering the pro¬ 
duction phase. Its purpose is to verify design accuracy to 
the maximum extent practicable at this stage of the life 
cycle. 

In software development, an alternative to the normal 
system life-cycle development approach is an evolutionary 
design method called "prototyping." Software prototyping 
is particularly useful when requirements are difficult to 
specify in advance, when requirements may change signifi¬ 
cantly during development, or when user-developer com¬ 
munications are indirect or difficult. It is usually easier for 
users to tell what they do, or do not, like about an existing 
prototype than it is to express what they would like in an 
imaginary, undeveloped system. A. Milton Jenkins, Univer¬ 
sity of Wisconsin, defines prototyping as a four-step pro¬ 
cess to: (1) identify the user's basic requirements, (2) develop 
the initial prototype, (3) let the user use the prototype to 
refine requirements, and (4) revise and enhance the proto¬ 
type. Steps (3) and (4) are normally an iterative process. 
When that process is performed early in advanced develop¬ 
ment, with the understanding that the prototype will be 
discarded after the requirements are refined and that no for¬ 
mal documentation will be delivered, the process is called 
"rapid" prototyping. 

I hope this brief discussion has contributed to your 
understanding of competitive prototyping, engineering and 
preproduction prototypes, and rapid prototyping. In a 
future issue, I will discuss the Navy Rapid Prototyping pro¬ 
cess for solving urgent fleet war fighting deficiencies. 


Mr. Brown is a Professor of engineering management in the 
Research Directorate at DSMC. 
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A CAPITAL CRISIS IN THE DEFENSE 
INDUSTRY? 

(A Summary of Three Recent Studies of Financial Health and 
Investment Attractiveness of the Industry) 

Dr. Fred Waelchli 


Defense material acquisition in this country is a joint undertaking of three colossal and diverse national institutions: the executive 
branch, the legislative branch and the defense industry. Needs and interests, institutional and personal, of participants in the acquisi¬ 
tion process vary. While there must be a substantial overlap of genuine interests among the three for the acquisition process to succeed, 
there are important interests that inevitably and properly diverge or conflict. This is a normal situation, and it is the legitimate func¬ 
tion of markets and politics to resolve these differences. 

A special problem arises, however, if independent actions of two parties combine unexpectedly to threaten serious injury to the third, 
and thus impair ability of the defense acquisition process to function. Three recent studies independently suggest this situation has 
happened—that a series ofspecific actions taken by the Department of Defense and the Congress between 1984 and 1987 have combined 
to severely threaten the health of the defense industry. This article summarizes these three studies. David Westermann’s article, ‘ ‘Govern¬ 
ment Contract Profit Policy and Defense, ” in the May-June issue of Program Manager, explored ramifications of problems 
highlighted here. 

Issues raised are serious. You should recognize, however, that these studies originated in the defense industry, and must be con¬ 
sidered, to some extent, advocacy journalism. No attempt is made in this article to be even-handed; views expressed are those of the 
participants in the studies as interpreted by the author. These views are not endorsed by the Defense Systems Management College, 
Program Manager, or any office in the Department of Defense. Program Manager solicits thoughtful, factual, concise, and 
readable expressions of opposing or divergent views on issues raised in this article. 


I he U.S. defense industry is in genuine trouble, and 
JIL its problems are deepening rapidly. This is the 
ominous theme of three recent studies of the defense in¬ 
dustry, one released in July 1987, and the other two in 
February 1988. 

The studies' common assertion is that a handful of un¬ 
coordinated legislative and administrative actions, taken in 
government in the last five or so years, has fundamentally 
altered the risk/reward relationship in the defense industry. 
These actions have simultaneously raised defense contrac¬ 
tors' risks and lowered their prospective rewards so that risk- 
adjusted returns on assets invested in the defense industry 
are no longer competitive with the range of returns available 
to capital invested elsewhere. Therefore, we will see (in fact, 
already see) a significant capital flight from the defense in¬ 
dustry, with serious consequences for national security. 
Provenance of Three Studies 
The first study was by the Financial Executives Institute 
(FEI) a professional organization of senior financial and ad¬ 
ministrative offices in U.S. business organizations. The 
second study was by 13 senior defense industry executives 
chartered in August 1987 as the ad hoc Defense 
Industry Advisory Group (DIAG) by the Defense Industry 


and Technology Subcommittee of the Senate Armed Ser¬ 
vices Committee (SASC). The third was commissioned by 
three industry associations. National Security Industrial 
Association, Aerospace Industries Association, and Elec¬ 
tronic Industries Association, and was performed by the 
MAC group.* 

A Mare's Nest of Government Acquisition Actions 

Each study cited the following governmental actions as 
significant contributors to an incipient exodus of capital from 
the defense industry. You should understand what follows 
is, in each case, a terse and necessarily simplified summary 
of a complex action or situation. 

Special Tooling and Test Equipment Investment 
Recovery. The Fiscal Year 1987 Defense Appropriation Act 
(P.L. 99-591) required contractors to capitalize and amor¬ 
tize (rather than expense) costs of investment in special 
tooling required to make unique products, and costs of 
special test equipment. The FY 88-89 Defense Authorization 
Act (P.L. 100-180) wrote into permanent law the require¬ 
ment for contractor investment in 50 percent of special tool¬ 
ing costs. Legislation makes payment of profit of these in¬ 
vestments a matter of governmental case-by-case 
discretion. 
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Reduced Rate of Progress Payments. 
The Fiscal Year 1987 Defense Ap¬ 
propriation Act (P.L. 99-591) reduced 
progress payments on defense con¬ 
tracts from 80 percent to 75 percent. 
The earlier rate had been 90 percent or 
80 percent, dependent on size of the 
firm. 

Restrictions on Completed Contract 
Method of Accounting. The 1983, 
1986, and 1987 tax-law revisions suc¬ 
cessively reduced ability of the con¬ 
tractor to defer recognition of, and 
payment of taxes on, contract profit 
until completion of the contract. This 
change results in earlier tax payments, 
with a consequent negative cash flow 
and a new need to finance vanished 
working capital funds. 

One Percent Reduction in Profit 
Margin Markups. The Fiscal Year 1987 
Defense Appropriation Act (P.L. 
99-591) reduced pre-negotiation profit 
margin markups by a target of 1 per¬ 
cent of costs or about 10 percent of 
profits. The law partially offset these 
reductions with variations designed to 
reward a contractor for investment in 
facilities and capital investment. 

Lower Cost Recoveries. The Con¬ 
gress has capped the amount of In¬ 
dependent Research and Development 
(IR&D) and Bid and Proposal (B&P) 
costs the contractor can recover 
through overhead billing on govern¬ 
ment contracts, and has prohibited the 
payment of profit on IR&D. The Con¬ 
gress added to the list of normal and 
unavoidable costs of doing business 
that are decreed to the "unallowable" 
for contract reimbursement pur¬ 
poses. 

Transfer of Rights in Duiu to 
Government. The Fiscal Year 1985 
Defense Authorization Act (P.L. 
98-525) required the Department of 
Defense to issue regulations defining 


the government's legitimate interests in 
contractor technical data rights. Com¬ 
plaints of Department of Defense 
alleged overuse of data rights provi¬ 
sions apparently provoked new legisla¬ 
tion (P.L. 99-961; FY 1987 Defense 
Authorization Act) stating the congres¬ 
sional belief that Department of 
Defense regulations required contrac¬ 
tors to give up too much proprietary 
information, and directing Defense to 
rework regulations. 

Research and Development Cost 
Sharing. As a matter of common prac¬ 
tice, contractors are frequently re¬ 
quired to pay part of the contract 
research and development costs of new 
systems, with no guarantee of a 
follow-on procurement contract in 
which to recoup the investment. 

Fixed-Price Development Contracts. 
While not a law or Department of 
Defense policy, it had become custom 
for some military services to insist on 


fixed-price contracts for research and 
development of new systems, and to 
demand fixed-price production options 
before the end of product develop¬ 
ment. The effect has been to shift 
sizable, usually uncompensated, risk to 
the contractor. There has been recent 
action on this issue; the FY 1988 
Defense Appropriation Act restricts 
Department of Defense use of fixed- 
price development contracts and re¬ 
quires such contracts to be approved 
by the Undersecretary of Defense for 
Acquisition. 

SUMMARIES OF THREE 
REPORTS 

Financial Executives Institute 
Report, The Impact of Government 
Policies on Defense Contractors, July, 
1987. This report uses the device of the 
cumulative net cash flow diagram to 
illustrate notionally effects of cited 
governmental actions. Figure 1 shows. 




















FIGURE 2. EFFECT OF GOVERNMENT ACTIONS 
ON PROJECT NET CASHFLOW 



Source: Adapted from FEI Report 


Defense Industry Advisory 
Group Report to Subcommittee on 
Defense Industry and Technology, 
Senate Armed Services Committee, 
February 5, 1988. The report is the 
broadest, dealing with acquisition 
issues beyond the eight governmental 
actions cited here, but also addressing 
those actions. The Defense Industry 
Advisory Group was formed in 
August 1987 by Senator Jeff 
Bingaman, chairman, and Senator Phil 
Gramm, ranking minority member, to 
advise the Senate Armed Services 
Committee Defense Industry and 
Technology Subcommittee on 
. .those aspects of the acquisition pro¬ 
cess that stifle innovation, drain good 
talent away from the acquisition 
system, and threaten our technological 
lead." 


for an actual 1970 defense program, 
unnamed, but represented as typical, 
an initial gentle rate of negative cash 
flow during research, development, 
and initial production, which deepens 
sharply during early volume produc¬ 
tion, turns up as cash from sales ar¬ 
rives, and reaches positive territory 
(break-even) toward the end of 
production. 

Figure 2 shows effects the Financial 
Executives Institute asserts cited 
governmental actions would have on 
cash flow for the same project were the 
project to start today. Early negative 
cash flow is deeper, requiring the con¬ 
tractor to finance larger sums for 
longer periods, extending break-even 
date, and reducing final net positive 
cash balance. Were cash flows dis¬ 
counted, as is always the case in prac¬ 
tice, these effects would be magnified 
since negative cash flow is early and 
positive cash flow is late. 


The Financial Executives Institute 
report does not quantify effects it 
describes, but makes the following 
predictions based on its analysis: 

"Participation in defense contracting 
by both large companies and smaller 
specialty companies will be reduced as 
investment opportunities outside the 
industry become more attractive. The 
defense industrial base will shrink and 
competition will decline because the 
entry level investment has increased 
without a corresponding increase in 
the potential return. 

"Internationally, the competitive posi¬ 
tion of U.S. contractors will erode, 
with a corresponding negative impact 
on the U.S. balance of payments. 

"Finally, but most importantly, the 
worldwide technology lead of the U.S. 
defense industry may not be 
maintained." 


Chairman John Rittenhouse, 
General Electric Aerospace Corpora¬ 
tion, and the panel identified 20 issues 
in three thematic areas which they 
believed adversely affected ability of 
the current defense acquisition process 
to provide for needs of the armed 
forces, and its ability to ensure con¬ 
tinued advancement of technology and 
industrial productivity necessary for 
national security. 

The first Defense Industry Advisory 
Group theme was "People: the Pro¬ 
curement Workforce, and Organiza¬ 
tional Relationships within DOD" 
(two issues); second, "Process: 
Streamlining and Stabilizing the Pro¬ 
cess, Including Issues of Contractor 
Financing and Investment" (14 issues); 
finally, "Trust: the Relationships 
among Congress, the Executive 
Branch, and Industry" (four issues). 
The first four problem statements of 
the "process issues" center on the cited 
governmental actions and raise the 
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problem of capital flight from the in¬ 
dustry. Those four issues are stated as 
follows in the Defense Industry Ad¬ 
visory Group study: 

“Issue 1: The Conflict Between Profit 
and Investment Policies. The issue is 
that government acquisition policy 
seeks to create incentives for contrac¬ 
tors to make investments which benefit 
the government and to provide reason¬ 
able profits from which such in¬ 
vestments can be made. This policy 
became confused over the last two 
years as a result of conflicting 
legislative and regulatory initiatives. 
Without resolution of the conflict, this 
policy will fail. 

"Issue 2: Profits and Costs. Financial 
policies enacted in recent years to 
reduce DOD outlays and the federal 
deficit will increase procurement costs 
and reduce national security in the 
long term. Profits account for less than 
10% of defense expenditures, while 
contract costs account for more than 
90%. Many of these costs are the result 
of unnecessary government regula¬ 
tions. Attention should focus on non¬ 
productive costs. 

"Issue 3: Government Policy on In¬ 
dependent Research and Development 
(IR&D). For the past several years. 
Congress has established ceilings on 
Government-allowable, Independent 
Research and Development and Bid 
and Proposal (B&P) costs. These 
ceilings inhibit industry investment in 
advanced technologies and new 
products. 

"This year. Congress will consider 
whether to alter or eliminate the 
Government's traditional support for 
independent research and develop¬ 
ment. Because IR&D is at the founda¬ 
tion of the technological superiority of 
our armed forces, the IR&D system 
should be strengthened as an invest¬ 
ment in the future. 

"Issue 4: Shifting Undue Risk to the 
Contractor. Disregarding the lessons 
learned from failures of similar pro¬ 
curement methods in the past, the 
DOD is now employing procurement 
methods which shift unmeasurable 
risks to contractors in three different 
ways. First, contractors are being re- 
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quired to pay a portion, sometimes 
substantial, of the development costs 
of Defense Department systems, under 
a practice called 'cost sharing' . 
Second, contractors are being required 
to enter into fixed-price contracts early 
in development, when the uncertain¬ 
ty is so substantial that it is virtually 
impossible to know the precise costs of 
new systems. Third, contractors are 
being asked'to provide the Defense 
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Lhe domestic 
defense industry is 
no longer an attrac¬ 
tive investment; 
reward no longer 
compensates ade¬ 
quately for risk, and 
capital is fleeing. 
Strength in the 
defense industry is 
migrating offshore. 


defense business. How can this balance 
be restored? 


"Current policies discourage invest¬ 
ment in technology and productivity. 
How should they be changed? 


Department with priced production 
options before full-scale development 
has begun.... All of these requirements 
shift undue risk to the contractor, 
drain industry resources from in¬ 
vestments in technology and produc¬ 
tivity, and will ultimately affect our 
nation's ability to maintain 
technological superiority." 


The Defense Industry Advisory 
Group Report poses three "...fun¬ 
damental questions of how DOD 
should do business," derived from con¬ 
sideration of the cited set of govern¬ 
ment actions: 


"The most essential business equation, 
in which measurable risk is balanced 
by profit potential, no longer describes 


"Small manufacturing firms are forced 
to pay a high price for participating in 
the government market by having to 
give up the fruits of their innovation. 
What effect does this have on competi¬ 
tion?" 



The Defense Industry Advisory 
Group Study is qualitative; like the FEI 
study, it does not attempt to quantify 
the effects it describes. It asserts keys 
to improving the acquisition process 
are "...streamlining and program 
stability." 35. 

The MAC Report, The Impact on 
Defense Industrial Capability of 
Changes in Procurement and Tax 
Policy, 1984-1987, dated February, 
1988. This report is the only quantified 
analysis among the three studies. It 
takes the cumulative net cash-flow 
methodology used in the Financial Ex¬ 
ecutive Institute Report and applies it, 
with new procurement rules, to actual 
data from nine historical Department 
of Defense acquisition programs, all 
but one profitable, and each managed 
by a different defense contractor. The 
intent was to estimate effects of 
governmental actions on contractor 
cash flow, now and in the future. Like 
the Financial Executive Institute 
Report, the idea was to estimate finan¬ 
cial impacts on past programs if the 
new rules had been in effect. Among 
their findings: 







"The return on investment on the pro¬ 
grams analyzed would have been less 
than the return necessary to preserve 
shareholder value. Put simply, there 
would have been no financial reason 
to bid the programs. 


"Profits on current work will be 
substantially reduced—by an average 
of 23% on the companies' defense 
business. 


"Companies will be forced to borrow 
heavily, but the additional financing 
required will, for some companies, 
likely exceed the amount that can be 
borrowed. 
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“As companies feel the squeeze on 
available capital, they will be forced 
to: 

—Reduce (already declining) con¬ 
tractor-funded research and develop¬ 
ment investment. 

—Reduce risk by opting for low- 
technology alternatives. 

—Reduce investment needed for 
productivity enhancement and 
modernization. 

—Decline to bid programs with un¬ 
realistic provisions. 

"While the full financial impact may 
not be reflected in the industry's finan¬ 
cial statements for three, or even more, 
years, the industry is already be¬ 
ginning to respond in these terms. The 
financial impact will probably show 
first in subcontractor financial 
statements. 

"The governmental changes will result 
in a less efficient industry. Cutting 
back on capital investment is mort¬ 
gaging the future, slowing down the in¬ 
dustry's ability to improve production 
efficiency. 

"Competition will be reduced because 
there will be fewer competitors willing 
to bid on future programs." 


Dismal Message 

The composite message of three 
studies is dismaying. The domestic 
defense industry is no longer an attrac¬ 
tive investment; reward no longer 
compensates adequately for risk, and 
capital is fleeing. Strength in the 
defense industry is migrating offshore. 
If the present trend goes unchecked, 
the U.S. defense industry will shrink, 
competition will decline, technological 
progress will stagnate, and the quest 
for preeminent performance in U.S. 
weapon systems will falter as industry 
declines to accept the prerequisite 
development risks. 

No one planned for such a situation, 
and nobody wants it. Why is it now 
a real prospect7 



The Predicament 
If we accept the validity of the three 
studies, the mistake leading into the 
current predicament was a series of 
nearly simultaneous, uncoordinated 
actions by the Congress and the 
Department of Defense, each logical¬ 
ly designed for a limited objective, but 
whose sum foreshadows a possible na¬ 
tional security nightmare. 


The way out, 
in direct contrast to 
the way in, must be 
carefully and 
thoughtfully 
planned and 
coordinated by 
those with power 
to make changes, 
the Department of 
Defense and the 
Congress. 


The first lesson, therefore, is that ac¬ 
quisition is, or takes place within, a 
complex social system. Complex social 
systems can be controlled only by well- 
designed and properly crafted control 
and administrative devices; "simple" 
control devices do not control and in¬ 
evitably produce unanticipated, 
usually unwelcome, side effects. 

The way out, in direct contrast to 
the way in, must be carefully and 
thoughtfully planned and coordinated 
by those with power to make changes, 
the Department of Defense and the 
Congress. Proposed actions, singly 
and in combination, must be tested for 
impact on the system before they are 
imposed. The MAC Report 
argues: 


"...Congress should ensure that the 
type of uncoordinated policy change 
that led to this unintended utcome is 
not repeated. The Congress and DOD 
must undertake some form of impact 
assessment...before introducing 
policies whose cumulative impact on 
the industry is as severe as we have 
shown. 

"...Congress and DOD should develop 
a clearer vision of the type of defense 
industry that will be required in five 
to ten years. It should test its procure¬ 
ment policies against these re¬ 
quirements, and ensure that they pro¬ 
vide an incentive to invest in the 
production facilities and equipment, 
research, and human resources that the 
industry and the DOD will 
require." 

In discussing the Defense Industry 
Advisory Group Report with the 
Senate Armed Services Committee 
staff, a Defense Systems Management 
College faculty team proposed com¬ 
missioning a major study to examine 
legitimate needs and interests of the Ex¬ 
ecutive branch, the Congress and in¬ 
dustry to see if a viable, "win-win," 
modus vivendi, supportive of national 
security and acceptable to all 
stakeholders (some used the word 
"treaty"), could be devised. 

Clearly, there are different ways to 
approach this problem. Surely we need 
to start now to construct a reasoned 
plan to ensure that we will never again 
find ourselves unwitting victims of 
fragmented, uncoordinated, contradic¬ 
tory, and ultimately damaging acquisi¬ 
tion policy and regulations. 

As this article is written, there are 
signs the capital flight problem is 
becoming visible to the Congress and 
the Department of Defense. Senator 
Bingaman's subcommittee on the 
defense industrial base heard testimony 
on two of the three reports I have 
reviewed. The Department of Defense 
appears to be backing off on fixed- 
price development contracts and 
shows evidence of second thoughts on 
cost sharing. By now, there may be 
other changes affecting the capital- 
flight problem. 

As yet, however, there is no sign of 
a coordinated, thoughtful, strategic ap- 
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proach toward defining or crafting the 
optimum or desired relationship be¬ 
tween government and the defense in¬ 
dustry. There is every evidence that we 
are proceeding to "piecemeal" our way 
out of the predicament just as we 
"piecemealed" our way in. Thus, we 
guarantee no reasoned or long-term 
systemic solution to the structural 
problem. 

Specific Approaches to 
Capital-Flight Crisis 

What can be done to resolve today's 
apparent defense industry capital flight 
crisis? Here is a portion of the MAC 
Report's response: 

"Our study makes explicit an issue that 
DOD procurement policy must face. 
Business is fundamentally about risks 
and returns. As essentially the only 
purchaser of highly specialized defense 
equipment, DOD controls both sides 
of the risk,'return balance (at least for 
major systems procurement). In the 
period we have examined, DOD and 
the Congress decided to adjust what 
was viewed as an imbalanced risk- 
return relationship. Unfortunately 
[they] reduced rewards and increased 
risks simultaneously, with not one, but 
multiple, uncoordinated adjustments. 
At the same time. Congress signifi¬ 
cantly increased the industry's capital 
requirements (by reducing progress 
payments and deferred tax 
financing). 

"While some in DOD now claim the 
industry is much more like commercial 
industry. Wall Street is saying it will 
not provide it with capital at the same 
rate as commercial industry. Wall 
Street might provide the capital if it 


saw the opportunity for high returns 
(as it does for biotechnology com¬ 
panies, for example); the industry 
might live with low profits if the 
government provided more of the 
financing and did away with cost shar¬ 
ing, fixed-price development, and 
other unreasonable risks. But, as mat¬ 
ters currently stand, government has 
stepped out. Wall Street is unwilling to 
step in, and the industry is unable 
to. 

"Solutions have to address both prof- 
fitability and risk, which have to be in 
balance. This introduces the question 
of what is the appropriate 
balance?" 

What is the appropriate and 
politically acceptable balance between 
profitability and risk in the defense in¬ 
dustry? That seems to be the question 
to answer soon in a coordinated way 
by everyone concerned. 

Bottom Line for Industry 

Individual governmental actions in 
the three reports are complicated, and 
contributions of each action to the 
general problem are difficult to assess, 
as each study makes clear. The overall 
issue, however, could not be simpler. 

As a nation, we choose to acquire 
military material and equipment 
through a privately financed, private¬ 
ly managed, free, and non-coercive 
market system. In this system, pri¬ 
vately owned assets are invested to ob¬ 
tain a return, and these assets flow in¬ 
exorably to investments appearing to 
offer the best risk-adjusted returns. An 
investment offering a less-than- 
competitive, risk-adjusted return will 
not attract capital. 


The common theme of the three 
studies is the effort of the government, 
in the eight issue areas and elsewhere, 
to chip away at, to limit, and to lower 
the risk-adjusted return allowed on 
private assets invested in defense in¬ 
dustry, hoping to lower weapon ac¬ 
quisition costs. If these actions com¬ 
bine to force the aggregate risk- 
adjusted return on defense work below 
the competitive range, as some argue 
is already the case, the result must be 
the flight of capital, new and replace¬ 
ment, from the defense industry— 
surely an intolerable prospect for na¬ 
tional security. 

Copies of the studies discussed in 
this article may be obtained as follows: 

—FE1 Report: Financial Executives 
Institute, Washington Office, 
1100 Seventeenth Street, N.W., 
Washington, D.C. 20036. 67. 
—DIAG Report: Subcommittee on 
Defense Industry and Technology, 
Senate Committee on Armed Services, 
United States Senate, Washington, 
D.C. 20510. 68. 

—MAC Report: National Security In¬ 
dustrial Association, National Head¬ 
quarters, 101515th Street, N.W., Suite 
901, Washington D.C. 20005. 69. 

‘The MAC Group is a consulting firm 
formed by a group of Harvard 
Business School professors. 


Doctor Waelchli is a Professor of manage¬ 
ment, Business Management Department, 
at the Defense Systems Management Col¬ 
lege and a member of the DSMC Center 
for Acquisition Management Policy. 
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PROGRAM MANAGER’S GUIDE 

FOR MONITORING CONTRACTOR 
OVERHEAD/INDIRECT COST 

Ronald L. Baker 


II n December 1984, the Deputy Secretary of Defense 
-Jfc- issued a memorandum, "Initiative to Reduce 
Overhead." The memorandum identified ten principles of 
overhead cost control and was forwarded to chief executive 
officers of major defense firms seeking their help in con¬ 
trolling overhead costs. These principles gave government 
and industry objectives to monitor and manage overhead 
cost and were intended to set the direction and tone of the 
Department of Defense (DOD) commitment to overhead 
reduction. The ten principles of overhead cost control ap¬ 
peared in the March-April 1985 issue of Program 
Manager. 

Since then, government and industry have done much to 
focus on reducing overhead/indirect costs on weapon 
systems. Developing improved overhead cost-reduction pro¬ 
cedures is increasingly important because overhead is a ma¬ 
jor percentage of total contractual costs. Increased 
mechanization produces an increasing overhead percentage 
in the absence of strong control measure. A major reason 
for this increase is the requirement for highly skilled per¬ 
sonnel to maintain computer hardware and software. Pro¬ 
grams such as the Industrial Modernization Incentives Pro¬ 
gram (IMIP) reduce direct cost elements through capital in¬ 
vestment. Without overhead reduction, portions of the 
potential total cost reduction will be lost. 

Through the IMIP, the DOD provides contractors incen¬ 
tives to invest in capital improvements. This improves pro¬ 
ductivity and reduces costs. "Productivity Improvements to 
Reduce Contractor Overhead," a research paper completed 
in April 1987 by an Air Force Reserve group assigned to the 
Aeronautical Systems Division, indicated many techniques 
and concepts used in the IMIP program to identify changes 
and provide incentives to contractors have direct applica¬ 
tion to overhead reductions. Indirect or overhead costs are 
always significant or dominant in the cost of systems ac¬ 
quisition. These costs are allocated to all programs in a con¬ 
tractor’s plant. This makes it difficult for program managers 
to apply control mechanisms. This multiple-program aspect 
of overhead is similar to contractors' capital facilit> -*• and 
equipment which normally benefit their programs, direct 
program costs and indirect overhead costs share most ex¬ 
penses; i.e., labor, equipment, and materials. As such, in¬ 
vestments could reduce direct and overhead costs and im¬ 
prove productivity. 

Currently, overhead cost is controlled by the Forward 
Pricing Rate Agreement (FPRA). This utilizes provisional 
billing rates based on historical experience and estimates of 
business volume during the period involved. Final rates are 
negotiated after the end of the period and an audit of ac¬ 
tual overhead costs. The FPRA approach has been in place 
for the past 15 years. One study states it "emphasizes three 




The current Air Force system for overhead control uses 
FPRAs primarily, but includes the Monitoring Overhead 
through Discrete Evaluations (MODE) computer program; 
overhead should-cost evaluations; the Air Force indirect cost 
monitoring team consisting of an overhead monitor, an in¬ 
dustrial engineer and an auditor; and the proposed use of 
traditional fee-sharing to control overhead expenditures. A 
recent approach is the use of Memoranda of Understanding 
(MOU) between the Air Force and contractors covering 
some voluntary measures that the contractors will take to 
reduce overhead. 

Recently, the Defense Systems Management College 
developed the Program Managers Guide for Monitoring 
Contractor Overhead/Indirect Cost. This was written for 
the non-financial manager and provides a succinct tool for 
program managers and acquisition staffs to use in their over¬ 
sight roles of weapon systems overhead costs. The guide in¬ 
cludes overhead cost topics such as; the definition of 
overhead and indirect cost, creation of indirect cost pools, 
compliance with cost accounting standards, and explana¬ 
tion of procedures and tools used by the indirect cost team 
in monitoring contractors. There is no single source of 
literature available to provide acquisition personnel with a 
complete overview of indirect/overhead costs. The guide 
provides this overview and gives the program manager a 
greater understanding on the role and responsibility of 
monitoring program costs. 

This guide will be available early in FY 89. 
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factors: the concept of cost avoidance, the Air Force assump¬ 
tion of responsibility for overhead management at plants 
for which it has responsibility, and a larger role for the on¬ 
site plant office in the overhead process." Cost avoidance 
should focus efforts on future cost reduction. Also, the Air 
Force, rather than the Defense Contract Audit Agency, 
monitors and settles overhead matters and is involved in 
the on-site Air Force office in surveillance and settlement 
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Mr. Baker, formerly a member of the Research Staffat the Defense 
Systems Management College, is now Deputy Program Manager 
for Training Systems, U.S. Marine Corps Research, Development 
and Acquisition Command, Quantico, Va. 











PROCUREMENT 

REFORM 

INITIATIVES: 

COMPETITION 


I spoke to an Army group of ac¬ 
quisition leaders in July 1985, on 
the importance attached by the Con¬ 
gress to the Competition in Contract¬ 
ing Act (C1CA). Before addressing 
more recent procurement reform ini¬ 
tiatives, a word about competition. 

In 1982, there were skeptics within 
the Department of Defense thinking 
competition would cause a host of hor¬ 
rors; those outside the Department 
believed nothing could stop it from 
buying from favorite suppliers. So far, 
time has proved the skeptics 
wrong. 

Procurement actions awarded com¬ 
petitively have increased dramatically, 
from 48 percent in 1983 to 88 percent 
in 1987. The dollar value of com¬ 
petitively awarded contracts increased 
from 41.2 percent in 1983 to 58.8 per¬ 
cent in 1987. 

The vigil is not over. It is too soon 
to say everyone within the Department 
is responsible for acquisition- 
contracting officer, program manager, 
and those involved in requirements 
generation—necessarily believe com¬ 
petition is a good thing. That change 
in mind-set won't happen overnight. It 
won't occur unless the drive for com¬ 
petition is done in a sensible manner. 


The Honorable Bill Nichols 


As I said in 1985, soon after the 
Competition in Contracting Act 
passed, the Congress is looking for'a 
process in which people act as if money 
being spent is their own—and that they 
use common sense when spending that 
money. That means the person con¬ 
sider all available information and 
make a sound business judgment. 

One important aspect of sound 
business judgment is whether to utilize 
competition. Then, we believed peo¬ 
ple were not looking at benefits of 
competition, like reduced price, better 
quality, etc., because the system 
fostered an environment penalizing 
one for pursuing competition. It was 
safer and less time-consuming to ex¬ 
ecute a sole-source contract than to go 
competitive. 

By legislating changes in that pro¬ 
cess, I hope we removed some im¬ 
pediments. We created a presumption 
that competition made sense. We did 
not abolish, however, the need to 
make an informed judgment con¬ 
cerning the best acquisition 
approach. 

The fact is that virtually every pro¬ 
curement is different in some respect. 
No rule or regulation can be drafted to 
apply neatly and give the best result in 
every situation. So, no matter how 



Congressman Bill Nichots 
(D., Ala.) 


many rules there are, only you can 
assess the situation and structure the 
best acquisition strategy for the 
government. 

We are looking for personal ac¬ 
countability and elimination of un¬ 
necessary layers of bureaucracy, not to 
have a person to point the finger at 
when things go wrong; rather, to give 
authority back to the person respon¬ 
sible for making a decision, and to stop 
second-guessing. 

This concern was the genesis for the 
program executive officer (PEO) con¬ 
cept. Although this concept was con¬ 
ceived by the Packard Commission, 
the Congress was receptive to the idea 
because it fit hand-in-glove with 
another concern of ours—that there is 
no accountability in the Department of 
Defense acquisition process. Program 
managers were changing frequently, 
and so many people were involved in 
making programmatic decisions that it 
was impossible to say any one person 
was in charge of a program. 

By embracing the program executive 
officer concept we are trying to 
streamline the process by establishing 
simple lines of communication and 
responsibility. We hoped this might 
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force a reevaluation of the necessity for 
reviewing organizations. Even if it does 
not result in a reduction in the 
bureaucracy, however, we hope it will 
force people to make inputs to the 
decision-maker at the start, rather than 
second-guessing decisions already 
made. 

Some said various procurement 
reform measures take away authority 
of the contracting officer or program 
manager. I agree in some cases, when 
the Congress believed decisions being 
made were not appropriate, it has 
acted to limit that exercise of judg¬ 
ment. A good example is Navy fixed- 
price contracts for research and 
development. We understood what 
they were trying to accomplish but we 
did not believe using fixed-price con¬ 
tracts was the appropriate way to meet 
the objective. When this policy con¬ 
tinued despite objections, the Congress 
adopted a prohibition in the defense 
segment of the continuing resolution to 
use fixed-price contracts for research 
and development. 

Wfile the Congress has legislated 
many prohibitions on some acquisition 


No rule or 
regulation can be 
drafted to apply 
neatly and give the 
best result in every 
situation. 


activities, these prohibitions are, in 
reality, simple boundaries on the range 
of options available to the acquistion 
manager; prohibitions are in no way, 
shape, or form intended as a substitute 
for ability to examine facts and cir¬ 
cumstances and judge what is the best 
action. If you examine statements 
made by members of the Congress in 
the last few years, it should become ap¬ 
parent the consistent underlying theme 
is that people cannot and should not 
hide behind rules and regulations. The 
system cannot work otherwise. We 
need to make decisions based on 
uni ae circumstances of each procure¬ 
ment. The system relies on each per¬ 
son making the best decisions about 
each aspect of the acquisition 
process. 

The Congress knows the system is 
as good as the people who run it; thus, 
their concern about quality of the 
work force. In the last few years, there 
have been proposals to improve the lot 
of acquisition personnel. While I don't 
necessarily always agree with the ap¬ 
proach taken, you should know what 
these proposals entail. 

They include proposals to establish 
an elite acquisition corps, establish 
minimum educational criteria for ac¬ 
quisition personnel and, finally, 
eliminate established pay grades in 
favor of a system recognizing out¬ 
standing work with bonuses and merit 
pay. 

The Congress emphasized educa¬ 
tional opportunities for government 


T he Congress 
knows the system is 
as good as the people 
who run it; thus, 
their concern about 
quality of the work 
force. 


acquisition personnel. In 1987, we 
mandated that Department of Defense 
reassess acquisition training programs 
conducted by the military services and 
the Defense Logistics Agency. We 
established a requirement that training 
of acquisition personnel at least be 
coordinated within the Department of 
Defense. 

Some proposals are controversial. 
We may not have all the answers, but 
we are concerned. 

I hope you know the Congress 
recognizes there must be some judg¬ 
ment involved in the acquisition pro¬ 
cess. While we put boundaries on ex¬ 
ercising that judgment, we have at¬ 
tempted to make sure you retain the 
ability to make necessary decisions. 
Remember what an old Texan once 
said: "There's nothing in the middle of 
the road but yellow stripes and dead 
armadillos.” 


This is adapted from remark made at the 
Defense Systems Management College by 
The Honorable Bill Nichols, Democratic 
Congressman from Alabama, to the Army 
Competition Leaders Conference. 
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USE OF PRODUCTION RATE AS 
INDEPENDENT VARIABLES 
IN COST MODELS 


Dr. Michael N. Beltramo 
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An AIM-7 Sea Sparrow Missile is fired from the aft Basic Point Defense Missile System launcher on the Amphibious Assault Ship USS Iuv Jima, LPH-2. 


Ill here is a growing trend in cost analysis to incorporate 
-JIL. production rate variables into cost estimating rela¬ 
tionships intended for general application. An article in The 
Journal of Cost Analysis (Camm, Gulledge and Womer, 
"Production Rate and Contractor Behavior," Vol. 5, No. 
1) identifies key studies related to the effect of production 
rate on cost and provides an excellent quantitative analysis. 
These notes were written from a different perspective; 
specifically, hypotheses should not be accepted unless they 
can withstand theoretical and logical scrutiny as well as em¬ 
pirical validation. 

The following comments suggest logical hurdles that need 
to be cleared to accept a model that relates production cost 
to production rate for similar items produced in different 
environments. 


There are many possible ways to produce a particular item 
by making capital/labor and raw material/purchased part 
tradeoffs, for example. Each firm has unique characteristics 
driving its production processes. In particular, two general 
types of plants exist in defense industries. One is product 
oriented where each output has a dedicated production line 
including labor, tooling, and capital equipment. In many 
cases, a product-oriented plant produces a single item. The 
other is process oriented where a particular function like 
circuit-board production or testing is performed for end- 
items. The following assumptions may be made about op¬ 
timal production rates and plant types: 


—A discrete, optimal production ra 
item in a product-oriented plant. 


A conceptual overview of production rate and its pre¬ 
sumed effect on production cost gets to the heart of the mat¬ 
ter. There is general agreement that producing at an optimal 
rate results in the lowest cost, all else being equal. This 
allows available capital and labor, direct and indirect, to 
be fully utilized. At a lower rate, surplus is charged to the 
smaller output. At a higher rate, inefficient substitutions 
(e.g., added labor instead of additional equipment, or in¬ 
adequate maintenance) are made and increase cost. 


—An optimal production rate for a process-oriented plant 
depends on total through-put; that is, a decrease in produc¬ 
tion rate for one item may be compensated for by an in¬ 
crease for another with no decline in efficiency (i.e., increase 
in cost). 
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; may exist for a specific 


—Two product-oriented plants may have very different 
optimal-production rates for the same product depending 
upon how they are facilitized, staffed, and tooled. 
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TABLE 1. SPARROW AIM-7F BUY VS. 
PRODUCTION QUANTITIES 


FY 


RAYTHEON 

GENERAL DYNAMICS 


Buy 

Production 

Buy 

Production 

1972 

100* 




1973/74 

225 

24 

15* 

— 

1975 

600 

167 

70 

— 

1976 

880 

332 

210 

— 

1977 

1110 

925 

210 

70 

1978 

1400 

1086 

750 

395 

1979 

900 

1077 

1310 

391 

1980 

1144 

1000 

300 

1027 

1981 

— 

722 

— 

961 

1982 

— 

881 

— 

— 

1983 


41 



Total 

6359 

6255** 

2865 

2844** 


*Not included in production quantities. Note: First lots may be develop¬ 
ment units. 

**For Raytheon, the difference between the total buy and the total pro¬ 
duction quantity is because the production quantity does not include the 
first buy of 100 units and the FY 73/74 and FY78 production data sup¬ 
plied by NAVAIR were each missing two units. 

**For General Dynamics, the difference between the total buy and the 
total production quantity is because the production quantity does not in¬ 
clude the first buy of 15 units and the FY78 and FY80 production data 
supplied by NAVAIR were in net error of minus six units. 


The typical production cost vs. pro¬ 
duction '■ate relationship is charac¬ 
terized by a U-shaped curve. 
Therefore, a given rate may be too low 
for one plant to operate efficiently but 
too high for another. This is different 
than learning-curve theory which 
holds that "more is cheaper." Thus, the 
idea of correlating quantities by time 
period with cost for similar but not 
identical items produced in a variety 
of environments is difficult to fathom 
in the absence of other information. 

Even if the serious logical issues are 
ignored, there are practical problems 
related to calculating the effect of pro¬ 
duction rate on cost. The first problem 
is that rate data are not reported 
routinely. Therefore, annual procure¬ 
ment quantities are commonly used as 
a proxy for production rate because 
they are easier to obtain than data 
related to annual deliveries. Even 
deliveries may misrepresent when pro¬ 


duction actually occurred as prime 
contractor work may be completed but 
not delivered pending readiness of the 
government user to accept it or prime 
contractor labor could be essentially 
complete with delivery delayed pend¬ 
ing the integration of government fur¬ 
nished equipment; e.g., missile 
warheads or aircraft engines. 

The two may vary significantly. 
Consider annual procurement quan¬ 
tities and annual delivery quantities 
for Sparrow AIM-7F shown in Table 
1. Both firms used the time lag between 
contract award and the beginning of 
production to achieve a relatively 
stable production rate. Dramatic rate 
variances implied by annual procure¬ 
ment quantities that could have in¬ 
creased costs were avoided; therefore, 
any production rate effects calculated 
for that program based upon procure¬ 
ment quantities, rather than deliveries, 
may be more apparent than real. 
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Another problem observed when 
analysts apply production rate factors 
is that they do not adjust learning- 
curve slopes. This causes double 
counting. Production rate effects are 
the effect of quantity per time period 
on cost and learning is the effect of 
quantity on cost; therefore, learning 
curves implicitly include production 
rate effects, although they may be 
cancelled when they fluctuate 
significantly during the period 
covered. The need to adjust learning- 
curve slopes when applying a rate 
slope is obvious. An appropriate 
method for making the adjustment has 
not been identified. 

There is widespread agreement con¬ 
cerning the importance of production 
rate as a cost driver in the abstract. It 
is possible to develop cost estimating 
relationships which include an ap¬ 
propriate production rate variable for 
individual plants or specific products 
in a particular plant. It has not been 
demonstrated that various production 
rates for similar products in different 
production environments have, or 
should have, the same effect on cost. 


The author is president of Beltramo and 
Associates, a defense acquisition consulting 
firm in Los Angeles. 
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TOTAL QUALITY MANAGEMENT: 
A DOD EXAMPLE 

Commander J. C. Boudreaux, USN 


Q uality management 
principles are not 
new in the United States. 

Walter Shewhart wrote 
about control of industrial 
quality as early as 1931. 

Dr. W. Edwards Deming 
and Dr. J. M. Juran, two 
prominent American quality 
theorists, introduced quality 
management techniques to 
Japan in the 1950s. S. V. 

Feigenbaum, P. B. Crosby 
and others have translated 
general theories into prac¬ 
tical applications that are changing the corporate outlook 
of United States industry toward quality management. 
Although each individual quality theorist has a slightly dif¬ 
ferent emphasis, there are six basic principles to which all 
adhere. 

First, management, not the worker, is responsible for pro¬ 
cess quality. Management must provide resources and policy 
decisions to make process changes and facilitate tasks of the 
work force. 

Second, continuous improvement of process quality 
through control of variability yields productivity improve¬ 
ment. Quality management emphasizes continuous search 
for a better productivity by improving work methods and 
reducing rework. 

Third, the worker can provide a significant input to pro¬ 
cess quality control. The operator of a process is in the best 
position to identify process weaknesses and to offer sugges¬ 
tions for process improvement. 


Fourth, quality cannot be 
inspected into a product. In¬ 
spection identifies the defec¬ 
tive product but cannot 
eliminate cause of the defect. 

Fifth, statistical and 
graphic techniques support 
the quality management de¬ 
cision-making process. Sim¬ 
ple graphs define process 
capability and can be used 
by the manager to make de¬ 
cisions concerning changes 
to the process. 

Sixth, all levels of the organization must be trained 
to understand the company's quality philosophy and to im¬ 
plement that policy. Every level of an organization must 
have the common mission and common language with 
which to communicate. 

Resurgence of these principles has been stimulated by 
American industry's reaction to the Japanese competition 
in international markets. United States industry has begun 
using these total quality management principles to regain 
the competitive position in the world market. At least one 
group of government agencies is following industry's lead. 

A recent example of total quality management illustrates 
applicability of these quality principles to production and 
administrative areas within Department of Defense organiza¬ 
tions. The Naval Aviation Depot (NADEP), North Island, 
San Diego, Calif., is a Department of Defense organization 
making a long-term commitment to total quality 
management. 
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The NADEP is a complex industrial 
facility performing depot-level 
maintenance, overhauling and re¬ 
pairing engines and components from 
Naval aircraft, and performing similar 
maintenance on support systems 
related to naval aviation. The Navy 
operates six NADEPs at Norfolk, Va., 
Cherry Point, N.C., Jacksonville, Fla., 
Pensacola, Fla., Alameda, Calif., and 
San Diego, Calif. The six NADEPs 
represent a complex mix of products 
and processes. 

One early effort to improve quality 
control was an artisan certification 
program managed by the Quality 
Assurance Department at each 
NADEP. Previous programs included 
quality circles, quality goals, slogan 
programs, quality reporting and 
auditing activities. These programs 
contributed to quality awareness but 
fell short of providing coordinated, 
continuing improvement in the overall 
quality of produi t and in the efficiency 
of NADEP operations. 

By instituting quality management 
techniques, NADEP at North Island 
focused on increasing productivity and 
providing a product to meet customer 
expectations. The NADEP emphasized 
long-term management commitment 
to change corporate attitude. 
Previously, management approached 
change with fear and uncertainty. 
After total quality management prac¬ 
tices were implemented, management 
actively sought and encouraged 
change. 

Using quality management, the 
organization was trained at all levels 
to think in terms of process quality. 
The work force was taught to use sim¬ 
ple graphical methods of statistical 
process control to make statistically 
based decisions affecting their pro¬ 
cesses. The new attitude included 
mutual action among departments of 
the organization for process quality. 
Customer requirements were used to 
define good quality and the definition 
of customer was extended to include 
the user of products within and outside 
the organization. One management 
role in quality management was to 
assist each organization element to 
recognize internal and external 
customers and requirements. 

Program Manager 


The quality management program at 
NADEP in North Island evolved 
slowly during 4 years. Initial activity 
included exchanging books and 
discussing current industrial trends in 
quality management. Interest grew 
and, after visits to local industry, a 
small group of managers started an in¬ 
formal training program based on 
seminars of Dr. Deming. Between 
1980-84 the awareness and training ef¬ 
fort exposed 200 upper and middle 
managers to quality management 
ideas. A network developed to obtain 
and distribute literature concerning 
quality activity occurring in private in¬ 
dustry. Dr. Deming provided personal 
support to the network in the form of 
appearances at North Island to discuss 
quality theory. 

In 1984, the commanding officer of 
NADEP, North Island, commissioned 
a demonstration effort in the Manufac¬ 
turing Division of the facility. The 
demonstration attempted to establish 
implementation procedures and apply 
quality management techniques to the 
working environment. Specific objec¬ 
tive was to establish the customer-to- 
supplier relationship between the 
grinding and plating shops to reduce 
defects and reduce cost of correcting 
defects. By identifying and breaking 
down communication barriers between 
shops, the NADEP management 
started a continuous cycle of improve¬ 
ment in the processes supported by the 
two shops. The resulting cooperation 
between the shops was directed at 
building quality into the processes and 
eliminating the requirement for exten¬ 
sive inspection for defects. The initial 
training in group decision-making 
established common methods to ex¬ 
amine the process. Communications 
were sustained at regular meetings be¬ 
tween the shops. 

In addition to working-level 
meetings, shop supervisors met week¬ 
ly with the next level of supervision. 
This management team established 
priorities and assigned resources to 
working-level activities. This quality 
process was developed using guidelines 
from Kaoru Ishakawa's Guide to 
Quality Control but was uniquely 
adapted by the Naval Aviation Depot 
to its organization. 
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Any quality improvement system is 
a process and evolves by using tech¬ 
niques of quality management. The 
NADEP quality improvement system 
was systematically changed over a 
period of years. This dynamic manage¬ 
ment effort was successful in trans¬ 
forming the daily business routine 
from conflict to teamwork. In one 
specific case, combined efforts of the 
plating and grinding teams reduced the 
defect rate on a H-46 helicopter blade 
fold pin rework from 70 to 3 percent. 
Detailed techniques used to identify 
and eliminate contributors to the defect 
rate were part of the statistical process 
control methods of quality manage¬ 
ment. 

While the NADEP demonstration ef¬ 
fort continued in selected shops, upper 
management was taking plant-wide in¬ 
itiatives, using lessons learned from the 
demonstration. From the demonstra¬ 
tion effort, management realized that 
principles of process quality manage¬ 
ment applied to service and support 
processes as well as to production line 
processes. Concepts were successfully 
applied to such diverse areas as long- 
range planning activities and ad¬ 
ministration of individual travel 
procedures. 

The customer-supplier emphasis for 
service-oriented efforts were similar to 
the production effort. Each manage¬ 
ment group asked initial questions: 
What is our product? Who is our 
customer? Who are our suppliers? 

With answers to these questions the 
management used basic process quality 
management techniques to measure 
performance of the process that they 
managed. Management identified 
elements of the process they used to 
transform suppliers' inputs into the 
output for their customers. Each group 
identified and prioritized opportunities 
to improve their process and to 
enhance the quality of their ultimate 
product. 

Success of the improvement initially 
depended on energetic communica¬ 
tion. Communicating with suppliers 
ensured input specifications were met; 
communicating with process operators 
ensured the quality definitions support 
the ultimate product; and com¬ 
municating with the customer ensured 
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the product would meet the customer's 
expectations. Functions of "continuous 
process audit" and "control point 
sampling" were carried out contin¬ 
uously by management. Process audit 
activity was initiated by management 
using information from the customer 
or from workers that indicated an 
opportunity to improve the system. 
Control point sampling allowed 
management to monitor process fluc¬ 
tuations using measurements estab¬ 
lished and collected by the process 
operators at the working level. 
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tions into action brought about 
systematic changes to improve process 
quality. 


Once the process control system was 
in place, management continuously 
asked questions: How can we reduce 
variability in our product quality by 
reducing variability in our control 
points? Have conditions of our sup¬ 
plier's inputs changed? Have 
customers' requirements changed? 
Translating the answers to these ques¬ 


Principles demonstrated at North 
Island are applicable everywhere in the 
Department of Defense. By using this 
process control system, management 
can more closely control quality of 
output and avoid the quick-reaction 
crisis that leads to inefficiency. Advan¬ 
tages gained through quality control 
ultimately result in increased produc¬ 
tivity and reduced cost through 
elimination of process inefficiencies. 
The process control system applies 
regardless of the nature of the product. 
The concepts apply to all systems in 
the Department of Defense whether the 
product is hardware, software, paper¬ 
work or services. Ideas of customer-to- 
supplier interaction have tremendous 
potential for the Department of 
Defense. 


Capt. Ralph W. Ortengren, Jr., U.S. 
Navy, is Dean of the Department of 
Research and Information. His last 
assignment was Director, Strategic 
Sealift Division, Chief of Naval Opera¬ 
tions Staff. Captain Ortengren holds 
a B.B.A. degree and a B.S. degree in 
aeronautical engineering, both from 
the University of Michigan, and an 
M.S. degree in oceanography from the 
Naval Postgraduate School. He is a 
graduate of PMC 86-1 at the Defense 
Systems Management College. 

Gerald E. Keightly, Holder of Navy 
Chair, came to DSMC from the Navy 
Department and was Director, Plans 
and Programs, Strategic Systems Pro¬ 
gram Office. He received a B. A. degree 
in English literature from Villanova 
University, an M.A. degree in the same 
discipline from Georgetown Universi¬ 
ty, and an M.P.A. degree from the 
American University. 


Robert J. Ainsley is a Professor of 
educational research. Department of 
Research and Information. He came to 
DSMC from the U.S. Army Engineer 
School where he was Deputy Director 
for training and doctrine. He holds a 
B.A. degree from Emory and Henry 
College, an M.Ed. degree from 
Virginia State University, and M.A. 
and Ed.D. degrees, both from the 
Catholic University. 


Just as United States industry is 
being driven by competitive pressure 
to reexamine management techniques, 
the Department of Defense is being 
driven by budget reduction to improve 
productivity and increase cost effec¬ 
tiveness. The Department of Defense 
has taken specific actions to address 
specific productivity and cost- 
effectiveness symptoms. 

Total quality management 
represents a proved method that can 
be applied in all Department of 
Defense organizations to increase pro¬ 
ductivity and to reduce cost. Budget 
pressures demand action. Inaction will 
ultimately degrade the effectiveness of 
our defense system. 


Commander Boudreaux is assigned 
to the Naval Air Systems Command, 
Washington, D.C. 



William R. Fournier is a Professor of 
acquisition management in the School 
of Systems Acquisition Management. 
He had been a Project Manager for 
countermine projects. Mr. Fournier 
holds an A.S. degree in engineering 
from Holyoke Community College, a 
B.S. degree from the University of 
Massachusetts, and an M.B.A. degree 
from Southern Illinios University. 
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Donita J. McGeary is a Professor of 
educational research, Department of 
Research and Information. Previous¬ 
ly, she was Dean of Academics for the 
pilot course at the Army Management 
Staff College. Ms. McGeary earned a 
B.A. degree in education from the 
University of Maryland and the M.Ed. 
degree from Georgia Southern College. 
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Raymond W. Reig is a Professor of 
engineering management. Technical 
Management Department. He was 
assigned to Headquarters, Naval Air 
Systems Command, as Assistant Pro¬ 
gram Manager for Test and Evalua¬ 
tion. Mr. Reig received a B.S. degree 
from the U.S. Naval Academy, an 
M.S. degree from New York Univer¬ 
sity, an M.P.A. degree from Auburn 
University, and an M.B.A. degree 
from Southern Illinois University. 
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